-- Logs begin at Fri 2022-09-30 18:29:40 UTC, end at Fri 2022-09-30 19:53:18 UTC. -- Sep 30 19:14:48.848770 dv-cinder-860015-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 30 19:14:48.859441 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 30 19:14:48.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 30 19:14:48 2022] *** Sep 30 19:14:48.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 30 19:14:48.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 30 19:14:48.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: nodename: dv-cinder-860015-1-iscsi Sep 30 19:14:48.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: machine: x86_64 Sep 30 19:14:48.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: clock source: unix Sep 30 19:14:48.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: pcre jit disabled Sep 30 19:14:48.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: detected number of CPU cores: 4 Sep 30 19:14:48.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: current working directory: / Sep 30 19:14:48.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: detected binary path: /usr/bin/uwsgi-core Sep 30 19:14:48.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: your processes number limit is 47680 Sep 30 19:14:48.862175 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: your memory page size is 4096 bytes Sep 30 19:14:48.862175 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: detected max file descriptor number: 2048 Sep 30 19:14:48.862175 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: lock engine: pthread robust mutexes Sep 30 19:14:48.862175 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: thunder lock: enabled Sep 30 19:14:48.862175 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 30 19:14:48.862175 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 30 19:14:48.899534 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: Python main interpreter initialized at 0x561d4662e9c0 Sep 30 19:14:48.899534 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: python threads support enabled Sep 30 19:14:48.899534 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: your server socket listen backlog is limited to 100 connections Sep 30 19:14:48.899534 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: your mercy for graceful operations on workers is 90 seconds Sep 30 19:14:48.899860 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: mapped 403077 bytes (393 KB) for 2 cores Sep 30 19:14:48.899942 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: *** Operational MODE: preforking *** Sep 30 19:14:48.900199 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: *** uWSGI is running in multiple interpreter mode *** Sep 30 19:14:48.900199 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: spawned uWSGI master process (pid: 81390) Sep 30 19:14:48.900402 dv-cinder-860015-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 30 19:14:48.901894 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: spawned uWSGI worker 1 (pid: 81402, cores: 1) Sep 30 19:14:48.901894 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: spawned uWSGI worker 2 (pid: 81403, cores: 1) Sep 30 19:14:48.903311 dv-cinder-860015-1-iscsi devstack@keystone.service[81390]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 30 19:14:53.144889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 19:14:53.144889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: from cryptography.utils import int_from_bytes Sep 30 19:14:53.145684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 19:14:53.145684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: from cryptography.utils import int_from_bytes Sep 30 19:14:53.145684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 19:14:53.145684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: from cryptography.utils import int_from_bytes Sep 30 19:14:53.146248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 19:14:53.146248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: from cryptography.utils import int_from_bytes Sep 30 19:14:53.194572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 30 19:14:53.194731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 30 19:14:53.194817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 30 19:14:53.194891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 30 19:14:53.194958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] command line args: [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 30 19:14:53.195101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] command line args: [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 30 19:14:53.195194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 30 19:14:53.195333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 30 19:14:53.195420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ================================================================================ {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 30 19:14:53.195498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ================================================================================ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 30 19:14:53.195576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] admin_token = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.195744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] admin_token = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.195836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] config_dir = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.195974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.196054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_dir = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.196156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] config_source = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.196236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.196334 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] config_source = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.196446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] debug = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.196523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] debug = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.196594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.196796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.196894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.196963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.197099 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.197200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.197304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.197406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.197525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.197630 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.197757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.198375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.198473 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_config_append = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.198573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_config_append = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.198680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.198783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.198877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_dir = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.198977 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_dir = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.199078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_file = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.199184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.199299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.199414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.199497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.199613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.199694 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.199773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.199901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.199987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.200061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.200180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.200263 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.200347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.200493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.200577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.200651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.200776 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.200862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.200943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.201060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.201136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.201282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.201363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.201527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.201655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.201733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.201858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.201978 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.202062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.202245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.202331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.202722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.202838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.202942 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] publish_errors = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.203060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] publish_errors = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.203173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.203293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.203407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.203528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.203650 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.203771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.203890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.204012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.204134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.204255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.204389 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] standard_threads = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.204516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] standard_threads = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.204632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.204754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.204870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.204989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.205110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.205224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.205344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_journal = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.205525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] use_journal = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.205650 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_json = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.205767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] use_json = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.205887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_stderr = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.206008 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] use_stderr = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.206122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] use_syslog = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.206244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] use_syslog = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.206366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.206480 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 19:14:53.206594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.206707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.206822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.206944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.207060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.207180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.207301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.207416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.207529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.207651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.207767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.207884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.207998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.208111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.208224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.208347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.208477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.208601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.208722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.208844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.208958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.209080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.209080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.209303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.209425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.209540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.209654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.209780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.209903 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.210017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.210131 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.210251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.210366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.210488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.210609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.210723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.210837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.210958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.211072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.211263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.211380 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.211493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.211606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.211719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.211840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.211962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.212077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.212201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.212317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.212463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.212588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.212709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.212830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.212951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.213067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.213186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.213309 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.213424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.214148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.214270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.214391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.214506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.214629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.214744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.214859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.214979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.215101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.215217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.215340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.215455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.215576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.215691 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.215804 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.215950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.216076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.216193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.216307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.216435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.216550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.216664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.216826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.216945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.217059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.217172 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.217328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.217446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.217633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.217751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.217873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.218002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.218119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.218237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.218367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.218489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.218610 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.218725 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.218844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.218967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.219091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.219206 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.219320 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.219439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.219554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.219668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.219776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.219872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.219967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.220061 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.220154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.220249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.220356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.220513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.220612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.220739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.220844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.220970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.221086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.221200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.221315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.221429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.221550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.221550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.221783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.221899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.222021 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.222136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.222251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.222363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.222487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.222615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.222737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.222859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.222974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.223089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.223202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.223328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.223444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.223549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.223661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.223776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.223889 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.224027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.224180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.224304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.224425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.224555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.224671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.224795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.224911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.225015 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.225116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.225235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.225356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.225480 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.225597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.225712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.225826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.225940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.226054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.226177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.226293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.226414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.226530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.226645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.226759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.226881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.226996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.227110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.227224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.227338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.227452 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.227574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.227698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.227813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.227917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.228035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.228150 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.228272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.228445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.228564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.228679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.228794 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.228908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.229091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.229215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.229338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.229453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.229575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.229703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.229829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.229953 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.230070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.230184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.230306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.230422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.230537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.230660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.230776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.230892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.231006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.231121 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.231235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.231358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.231475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.231589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.231704 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.231818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.231933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.232057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.232173 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.232294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.232434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.232560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.232708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.external = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.232834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.232952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.233076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.external = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.233193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.233311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.233426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.233542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.233667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.password = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.233792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.233915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] auth.token = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.234033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.password = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.234150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.234265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] auth.token = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.234381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.234505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.234623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.234738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.234854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.234976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.235094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.235221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.235339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.235454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.235570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.235685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.235874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.235987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.236088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.236204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.236319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.236448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.236564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.236692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.236810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.236924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.237039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.237155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.237277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.237402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.237519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.237635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.237751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.237866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.237991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.238118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.238242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.238358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.238358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.238358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.238676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.238801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.238918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.239033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.239149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.239265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.239381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.239509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.239626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.239741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.239856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.239972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.240087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.240211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.240329 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.240511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.240630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.240746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.240862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.240989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.241107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.241223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.241339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.241454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.241578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.241695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.241811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.241926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.242041 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.242157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.242280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.242398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.242514 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.242629 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.242753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.242868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.243054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.243173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.243289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.243404 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.243520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.243642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.243767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.243884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.243998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.244114 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.244228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.244343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.244488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.244607 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.245225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.245344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.245344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.245561 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.245686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.245803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.245918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.246034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.246034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.246250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.246373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.246490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.246607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.246724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.246839 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.246954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.247083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.247200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.247317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.247433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.247433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.247657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.247782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.247899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.248018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.248134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.248134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.248350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.248493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.248610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.248733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.248850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.248966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.249082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.249206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.249324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.249440 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.249556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.249671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.249788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.249913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.250030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.250145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.250261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.250377 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.250492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.250616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.250733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.250850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.250966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.251088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.251220 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.251338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.251461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.251581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.251697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.251814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.251938 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.252055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.252170 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.252285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.252448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.252567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.252695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.252813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.253045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.253160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.253276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.253400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.253517 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.253633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.253748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.253864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.253980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.254104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.254221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.254337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.254453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.254569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.254684 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.254811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.254929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.255053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.255170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.255286 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.255401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.255525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.255643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.255758 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.255874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.255989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.256105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.256229 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.256347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.256477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.256592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.256708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.256833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.256959 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.257076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.257192 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.257307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.257422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.257538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.257661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.257778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.257894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.258019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.258137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.258252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.258376 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.258493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.258608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.258723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.258839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.258954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.259078 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.259197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.259313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.259428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.259543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.259657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.259781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.259897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.260013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.260128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.260245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.260477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.260613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.260732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.260848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.260964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.261079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.261201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.261328 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.261446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.261562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.261677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.261793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.261909 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.262033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.262150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.262266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.262382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.262497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.262612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.262736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.262853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.262969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.263074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.263188 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.263303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.263428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.263545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.263660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.263776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.263891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.264006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.264130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.264247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.264376 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.264487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.264601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.264716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.264843 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.264960 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.265075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.265190 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.265305 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.265420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.265544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.265661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.265776 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.265891 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.266006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.266195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.266314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.266430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.266546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.266662 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] role.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.266777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.266901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] role.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.267019 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.267134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.267249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.267364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.267479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.267604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.267721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.267837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.267953 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.268074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.268190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.268314 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.268475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.268593 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.268709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.268824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.268940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.269067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.269185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.269300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.269415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.269531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.269647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.269771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.269889 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.270005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.270122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.270238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.270353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.270477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.270595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.270711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.270711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.270929 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.271045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.271169 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.271287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.271402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.271518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.271634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.271759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.271877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.271993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.272108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.272224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.272340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.272485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.272603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.272719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.272835 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.272951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.273066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.273190 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.273307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.273422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.273538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.273654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.273769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.273894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.274012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.274127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.274243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.274358 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.274473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.274597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.274715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.274830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.274946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.275061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.275176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.275300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.275417 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.275532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.275647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.275762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.275878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.276520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.276639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.276755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.276870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.276987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.277102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.277226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.277342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.277457 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.277572 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.277687 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.277802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.277926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.278048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.278164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.278280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.278394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.278509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.278634 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.278751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.278866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.278989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.279121 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.279247 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.279364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.279480 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.279596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.connection = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.279724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.279841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.279956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.280071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.280199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.280317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.280488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.280607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.280723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.280839 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.280955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.281070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.281196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.281313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.281429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.281544 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.281660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.281775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.281899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.282017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.282132 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.282249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.282364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.282480 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.282604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.282721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.282837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.282952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.283068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.283184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.283309 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.283426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.283541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.283656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.283771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.283887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.284011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 19:14:53.284128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81402) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 30 19:14:53.284244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81403) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 30 19:14:53.308785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.308913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.309009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.309121 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.309206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.309346 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.317368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.317541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.345166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.345394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.345478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.345622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.397839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.398176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.447138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.447369 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.447581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.447835 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.601905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.602122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.602213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.602343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.602455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.602558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.602642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.602771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.602855 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.602981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.603075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.603190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.625757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.625988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.626073 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.626210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.626296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.626427 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.626511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.626643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.626737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.626857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.626942 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.627064 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.627147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.627271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.638574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.638733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.651446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.651623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.651708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.651847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.651933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.652067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.652150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.652285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.652398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.652522 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.652606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.652732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81403) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.652816 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.652943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81402) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 19:14:53.670005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.670212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 30 19:14:53.670548 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.670630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.670779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 30 19:14:53.671020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 30 19:14:53.671107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.671294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.671377 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 30 19:14:53.671552 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.671645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.671804 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.671887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.672051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.672132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.672301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.681553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.681830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.682094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.682424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.685971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.686247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.686514 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.686790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.689380 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.692260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.692563 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.692844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.693127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.693410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.693668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.693752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.693934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.694207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.694472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.694739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.695001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.696599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.696881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.697159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.697439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.697720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.698518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.698783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.699050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.699314 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.699585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.699845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.718061 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.719511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.719793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.722327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.723780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.724061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.724421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.724706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.724984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.728721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.729540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.729835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.731828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.732087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.736096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.736404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.737678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.737971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.738251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.738528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.738798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.739067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.739337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.739597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.741973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.742268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.742553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.742837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.743110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.743384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.743654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.743921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.755025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.755299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.759151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.759337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.759612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.762929 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.763221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.763993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.767258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.767402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.767550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.770629 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.771733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.774117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.774415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.774696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.774897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.774981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.775246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.775531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.778302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.778596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.778879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.779157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.779429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.779715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.792060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.792349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.792631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.792890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.796564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.796849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.797112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.797370 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.799665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.801212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.802725 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.802997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.804149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.805715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.805940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.806219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.806494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.806767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.807173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.807286 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.807554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.810351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.810632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.810914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.811190 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.811575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.818287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.818569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.818847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.819123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.819500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.819785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.820057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.822675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.823482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.823757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.824029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.824422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.824711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.824985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.835806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.838793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.840282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.841694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.843196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.843281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.844848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.845127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.845521 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.846083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.847571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.849048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.849326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.849722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.851809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.853276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.856633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.856741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.857100 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.857383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.857660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.858064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.860949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.861336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.861617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.861890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.865458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.865742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.866018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.869614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.869951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.870227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.873208 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81403) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.873495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.873772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.874046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.874315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.874591 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.874858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.875133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.875398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.875677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.875944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.876210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.876500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.876780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81403) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.877064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81402) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 19:14:53.877342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.877615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.877885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.878153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.878419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.878684 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.878953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.879215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.879489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.879763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.880026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.880284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.880570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81402) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 19:14:53.910961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561d4662e9c0 pid: 81403 (default app) Sep 30 19:14:53.914458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80583b9-40a8-4767-970c-b9b81a07c747 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:14:53.914739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80583b9-40a8-4767-970c-b9b81a07c747 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:14:53.914824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561d4662e9c0 pid: 81402 (default app) Sep 30 19:14:53.915007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80583b9-40a8-4767-970c-b9b81a07c747 None None] PATH_INFO: `/v3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:14:53.919784 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Fri Sep 30 19:14:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:00.627049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9adab1-7184-461d-9cea-b42a33ba734d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:00.627557 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9adab1-7184-461d-9cea-b42a33ba734d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:00.627557 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9adab1-7184-461d-9cea-b42a33ba734d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:00.631835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:15:00.637644 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b02a6-7792-4e31-bf41-8fe00bd5bd05 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:00.638089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b02a6-7792-4e31-bf41-8fe00bd5bd05 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:00.638232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b02a6-7792-4e31-bf41-8fe00bd5bd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:00.707156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: INFO dbcounter [None req-f82b02a6-7792-4e31-bf41-8fe00bd5bd05 None None] Registered counter for database keystone Sep 30 19:15:00.726317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG oslo_db.sqlalchemy.engines [None req-f82b02a6-7792-4e31-bf41-8fe00bd5bd05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81403) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 19:15:00.732968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writer thread running {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 19:15:00.911370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG passlib.handlers.bcrypt [None req-f82b02a6-7792-4e31-bf41-8fe00bd5bd05 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81403) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 30 19:15:00.912044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG passlib.handlers.bcrypt [None req-f82b02a6-7792-4e31-bf41-8fe00bd5bd05 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81403) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 30 19:15:00.969601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f82b02a6-7792-4e31-bf41-8fe00bd5bd05 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:00.971107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f82b02a6-7792-4e31-bf41-8fe00bd5bd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:01.098047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 30 19:15:01.424575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9adab1-7184-461d-9cea-b42a33ba734d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:01.492748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: INFO dbcounter [None req-8f9adab1-7184-461d-9cea-b42a33ba734d None None] Registered counter for database keystone Sep 30 19:15:01.513967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG oslo_db.sqlalchemy.engines [None req-8f9adab1-7184-461d-9cea-b42a33ba734d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81402) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 19:15:01.520094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writer thread running {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 19:15:01.637506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9adab1-7184-461d-9cea-b42a33ba734d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:01.641794 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2e8be-6e3a-4d28-8158-21afe729206a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:01.642412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2e8be-6e3a-4d28-8158-21afe729206a admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:01.642687 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2e8be-6e3a-4d28-8158-21afe729206a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:01.642960 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2e8be-6e3a-4d28-8158-21afe729206a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:01.651063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2e8be-6e3a-4d28-8158-21afe729206a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:01.703271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: WARNING keystone.common.rbac_enforcer.enforcer [None req-55c2e8be-6e3a-4d28-8158-21afe729206a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 30 19:15:01.703618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2e8be-6e3a-4d28-8158-21afe729206a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:01.712857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Fri Sep 30 19:15:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:15:01.718762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b02a6-7792-4e31-bf41-8fe00bd5bd05 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:01.735366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b02a6-7792-4e31-bf41-8fe00bd5bd05 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:01.739768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf5d36-4d0a-42e2-846a-2b380f8d99f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:01.740876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf5d36-4d0a-42e2-846a-2b380f8d99f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:01.741155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf5d36-4d0a-42e2-846a-2b380f8d99f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:01.741419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf5d36-4d0a-42e2-846a-2b380f8d99f4 admin admin] PATH_INFO: `/v3/projects/e574e273dabe4a3eab67dc95d42fcb6b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:01.742993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf5d36-4d0a-42e2-846a-2b380f8d99f4 admin admin] RBAC: Authorizing `identity:get_project(project_id=e574e273dabe4a3eab67dc95d42fcb6b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:01.796662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fdbf5d36-4d0a-42e2-846a-2b380f8d99f4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 30 19:15:01.797005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf5d36-4d0a-42e2-846a-2b380f8d99f4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:01.801413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Fri Sep 30 19:15:01 2022] GET /identity/v3/projects/e574e273dabe4a3eab67dc95d42fcb6b => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:03.910845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7d04-3300-4f77-bbd7-17d49659e92d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:03.911382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7d04-3300-4f77-bbd7-17d49659e92d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:03.911382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7d04-3300-4f77-bbd7-17d49659e92d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:03.915737 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:03.921669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b82222-4bdd-45a4-bfb7-7a146814a5f3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:03.922098 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b82222-4bdd-45a4-bfb7-7a146814a5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:03.922211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b82222-4bdd-45a4-bfb7-7a146814a5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:03.941658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-98b82222-4bdd-45a4-bfb7-7a146814a5f3 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:03.943144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-98b82222-4bdd-45a4-bfb7-7a146814a5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:03.950874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 30 19:15:04.217322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d7d04-3300-4f77-bbd7-17d49659e92d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:04.229758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d7d04-3300-4f77-bbd7-17d49659e92d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:04.234237 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77ff84-5b93-4859-a749-562988a64b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:04.234821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77ff84-5b93-4859-a749-562988a64b71 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:04.235104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77ff84-5b93-4859-a749-562988a64b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:04.235372 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77ff84-5b93-4859-a749-562988a64b71 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:04.242788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77ff84-5b93-4859-a749-562988a64b71 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:04.245243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77ff84-5b93-4859-a749-562988a64b71 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:04.253741 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Fri Sep 30 19:15:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:15:04.258920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b82222-4bdd-45a4-bfb7-7a146814a5f3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:04.267820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b82222-4bdd-45a4-bfb7-7a146814a5f3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:04.272087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef39ed-7ada-42db-a6b4-fda929cced1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:04.272733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef39ed-7ada-42db-a6b4-fda929cced1c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:04.273019 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef39ed-7ada-42db-a6b4-fda929cced1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:04.273292 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef39ed-7ada-42db-a6b4-fda929cced1c admin admin] PATH_INFO: `/v3/users/02c8dc74a8e74fac884b7eaf15083780` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:04.274605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef39ed-7ada-42db-a6b4-fda929cced1c admin admin] RBAC: Authorizing `identity:get_user(user_id=02c8dc74a8e74fac884b7eaf15083780)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:04.276416 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef39ed-7ada-42db-a6b4-fda929cced1c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:04.277913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Fri Sep 30 19:15:04 2022] GET /identity/v3/users/02c8dc74a8e74fac884b7eaf15083780 => generated 247 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:06.489749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ed510-5be3-438d-99c3-702e4bec4b75 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:06.490279 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ed510-5be3-438d-99c3-702e4bec4b75 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:06.490279 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ed510-5be3-438d-99c3-702e4bec4b75 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:06.491586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:15:06.496954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09c358-04a7-461e-8403-564268368b7c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:06.497377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09c358-04a7-461e-8403-564268368b7c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:06.497631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09c358-04a7-461e-8403-564268368b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:06.515955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4a09c358-04a7-461e-8403-564268368b7c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:06.516144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d7d99-38d1-4829-b1cb-a972180895cd None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:06.516280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d7d99-38d1-4829-b1cb-a972180895cd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:06.516575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d7d99-38d1-4829-b1cb-a972180895cd None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:06.517164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4a09c358-04a7-461e-8403-564268368b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:06.517916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:15:06.522892 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464c9e-5c37-4b49-8a8e-6602420e9e15 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:06.523174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464c9e-5c37-4b49-8a8e-6602420e9e15 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:06.523442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464c9e-5c37-4b49-8a8e-6602420e9e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:06.566258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG passlib.handlers.bcrypt [None req-86464c9e-5c37-4b49-8a8e-6602420e9e15 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81402) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 30 19:15:06.566644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG passlib.handlers.bcrypt [None req-86464c9e-5c37-4b49-8a8e-6602420e9e15 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81402) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 30 19:15:06.598649 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:06.615351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-86464c9e-5c37-4b49-8a8e-6602420e9e15 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:06.616983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-86464c9e-5c37-4b49-8a8e-6602420e9e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:06.714301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:15:06.865280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09c358-04a7-461e-8403-564268368b7c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:06.891084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:06.969552 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09c358-04a7-461e-8403-564268368b7c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:06.982289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86464c9e-5c37-4b49-8a8e-6602420e9e15 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:07.003244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b00f18-94c9-4a36-a962-0fee144554e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:07.003883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b00f18-94c9-4a36-a962-0fee144554e9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:07.004145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b00f18-94c9-4a36-a962-0fee144554e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:07.004438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b00f18-94c9-4a36-a962-0fee144554e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:07.016289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b00f18-94c9-4a36-a962-0fee144554e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-33b00f18-94c9-4a36-a962-0fee144554e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:07.026994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:07.027936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:07.028809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:15:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:07.033951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b00f18-94c9-4a36-a962-0fee144554e9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:07.082214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86464c9e-5c37-4b49-8a8e-6602420e9e15 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:07.084572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:07.115868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:07.120651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d4ee9-039a-45b8-a157-4fe4992fee25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:07.121510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d4ee9-039a-45b8-a157-4fe4992fee25 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:07.121792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d4ee9-039a-45b8-a157-4fe4992fee25 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:07.122351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d4ee9-039a-45b8-a157-4fe4992fee25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:07.133257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b00f18-94c9-4a36-a962-0fee144554e9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:07.167615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd65a0f-0229-4b1b-bd17-28c55a528659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:07.168211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65a0f-0229-4b1b-bd17-28c55a528659 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:07.168527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65a0f-0229-4b1b-bd17-28c55a528659 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:07.168791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65a0f-0229-4b1b-bd17-28c55a528659 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:07.169444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd65a0f-0229-4b1b-bd17-28c55a528659 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:07.169738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd65a0f-0229-4b1b-bd17-28c55a528659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:07.171107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd65a0f-0229-4b1b-bd17-28c55a528659 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:07.182422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:07.188015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd65a0f-0229-4b1b-bd17-28c55a528659 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:07.188132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d4ee9-039a-45b8-a157-4fe4992fee25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:07.189592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d4ee9-039a-45b8-a157-4fe4992fee25 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:07.285764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 30 19:15:07.286040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd65a0f-0229-4b1b-bd17-28c55a528659 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:07.291982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d4ee9-039a-45b8-a157-4fe4992fee25 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:07.294749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:07.320086 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948b6b7-bc13-4c0a-8baa-1356f46141ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:07.320731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948b6b7-bc13-4c0a-8baa-1356f46141ac None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:07.321008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948b6b7-bc13-4c0a-8baa-1356f46141ac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:07.321272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948b6b7-bc13-4c0a-8baa-1356f46141ac None admin] PATH_INFO: `/v3/users/02c8dc74a8e74fac884b7eaf15083780` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:07.322384 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948b6b7-bc13-4c0a-8baa-1356f46141ac None admin] RBAC: Authorizing `identity:get_user(user_id=02c8dc74a8e74fac884b7eaf15083780)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:07.323858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948b6b7-bc13-4c0a-8baa-1356f46141ac None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:07.325426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:07 2022] GET /identity/v3/users/02c8dc74a8e74fac884b7eaf15083780 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:07.330695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948b6b7-bc13-4c0a-8baa-1356f46141ac None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:07.345222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:07.390210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d4ee9-039a-45b8-a157-4fe4992fee25 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:07.423846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75b31f-01fb-49dd-b4dc-67510959f5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:07.424727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75b31f-01fb-49dd-b4dc-67510959f5ec None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:07.425005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75b31f-01fb-49dd-b4dc-67510959f5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:07.425271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75b31f-01fb-49dd-b4dc-67510959f5ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:07.432359 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948b6b7-bc13-4c0a-8baa-1356f46141ac None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:07.443052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75b31f-01fb-49dd-b4dc-67510959f5ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:07.444540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75b31f-01fb-49dd-b4dc-67510959f5ec None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-6e75b31f-01fb-49dd-b4dc-67510959f5ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:15:07.462656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:07.463653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:15:07.464525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:15:07.464525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:15:07.464525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:07.464525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:15:07.464750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357e304-8987-41e7-8753-7ac77bd61f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:07.464998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357e304-8987-41e7-8753-7ac77bd61f10 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:07.465266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357e304-8987-41e7-8753-7ac77bd61f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:07.465535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357e304-8987-41e7-8753-7ac77bd61f10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:07.466640 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357e304-8987-41e7-8753-7ac77bd61f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:07.468083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357e304-8987-41e7-8753-7ac77bd61f10 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:07.469771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75b31f-01fb-49dd-b4dc-67510959f5ec None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:07.469879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:07.475112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357e304-8987-41e7-8753-7ac77bd61f10 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:07.524481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:07.553821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:07.571188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75b31f-01fb-49dd-b4dc-67510959f5ec None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:07.572686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357e304-8987-41e7-8753-7ac77bd61f10 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:07.604684 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46670e5c-1e52-4104-a27e-51522bb99bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:07.605291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46670e5c-1e52-4104-a27e-51522bb99bca None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:07.605566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46670e5c-1e52-4104-a27e-51522bb99bca None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:07.605834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46670e5c-1e52-4104-a27e-51522bb99bca None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:07.606470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46670e5c-1e52-4104-a27e-51522bb99bca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:07.606555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f8031-578b-400b-ac6d-847a07c5db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:07.606769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46670e5c-1e52-4104-a27e-51522bb99bca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:07.607129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f8031-578b-400b-ac6d-847a07c5db67 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:07.607413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f8031-578b-400b-ac6d-847a07c5db67 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:07.607688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f8031-578b-400b-ac6d-847a07c5db67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:07.608140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46670e5c-1e52-4104-a27e-51522bb99bca None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:07.608437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f8031-578b-400b-ac6d-847a07c5db67 None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.domain.id=default, user.id=02c8dc74a8e74fac884b7eaf15083780) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:07.608746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f8031-578b-400b-ac6d-847a07c5db67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:07.610163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f8031-578b-400b-ac6d-847a07c5db67 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:07.616653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 11/20] 10.222.0.17 () {56 vars in 1322 bytes} [Fri Sep 30 19:15:07 2022] GET /identity/v3/role_assignments?user.id=02c8dc74a8e74fac884b7eaf15083780&scope.domain.id=default&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:07.626435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 10/21] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:10.077397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569ff3e-012b-4f33-8e87-774e71248797 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:10.077926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569ff3e-012b-4f33-8e87-774e71248797 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:10.077926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569ff3e-012b-4f33-8e87-774e71248797 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:10.079270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 12/22] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:10.092636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218b593-d7fa-4e62-aec0-73b8bdeff1ed None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:10.093089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218b593-d7fa-4e62-aec0-73b8bdeff1ed None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:10.093187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218b593-d7fa-4e62-aec0-73b8bdeff1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:10.117344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1218b593-d7fa-4e62-aec0-73b8bdeff1ed None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:10.118543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1218b593-d7fa-4e62-aec0-73b8bdeff1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:10.153543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e5f3-9918-45ed-9a4f-93c63c373bf7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:10.154165 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e5f3-9918-45ed-9a4f-93c63c373bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:10.154716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293e5f3-9918-45ed-9a4f-93c63c373bf7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:10.156581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 13/23] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:10.162688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c8e1e-f4be-4a2a-b87f-326e187c2b3c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:10.163289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c8e1e-f4be-4a2a-b87f-326e187c2b3c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:10.163754 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c8e1e-f4be-4a2a-b87f-326e187c2b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:10.197870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-737c8e1e-f4be-4a2a-b87f-326e187c2b3c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:10.199399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-737c8e1e-f4be-4a2a-b87f-326e187c2b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:10.246680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 11/24] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:10.305587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 14/25] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:15:10.618197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218b593-d7fa-4e62-aec0-73b8bdeff1ed None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:10.679344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:10.750569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218b593-d7fa-4e62-aec0-73b8bdeff1ed None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:10.795063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39da20e-eb5e-4454-8725-4a4cdc3405aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:10.795709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39da20e-eb5e-4454-8725-4a4cdc3405aa None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:10.795988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39da20e-eb5e-4454-8725-4a4cdc3405aa None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:10.796269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39da20e-eb5e-4454-8725-4a4cdc3405aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:10.811404 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c679e2-7480-406a-b5a7-06dffaf94e85 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:10.811685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c679e2-7480-406a-b5a7-06dffaf94e85 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:10.811961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c679e2-7480-406a-b5a7-06dffaf94e85 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:10.813510 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 15/26] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:15:10.820659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39da20e-eb5e-4454-8725-4a4cdc3405aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d39da20e-eb5e-4454-8725-4a4cdc3405aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:10.825124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:10.826098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:10.826996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:10.827600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bdfba-d70d-4bc5-b86b-2092a8fd5f46 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:10.827600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bdfba-d70d-4bc5-b86b-2092a8fd5f46 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:10.827600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bdfba-d70d-4bc5-b86b-2092a8fd5f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:10.829345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 12/27] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:15:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:10.830257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39da20e-eb5e-4454-8725-4a4cdc3405aa None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:10.862972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-282bdfba-d70d-4bc5-b86b-2092a8fd5f46 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:10.865015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-282bdfba-d70d-4bc5-b86b-2092a8fd5f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:10.946968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 16/28] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:10.948179 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bdfba-d70d-4bc5-b86b-2092a8fd5f46 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:10.951616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:10.953769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39da20e-eb5e-4454-8725-4a4cdc3405aa None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:10.972908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:11.015864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108fec3-da12-454b-8556-77295476f99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:11.016788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108fec3-da12-454b-8556-77295476f99c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:11.017201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108fec3-da12-454b-8556-77295476f99c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:11.017601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108fec3-da12-454b-8556-77295476f99c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:11.045397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108fec3-da12-454b-8556-77295476f99c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:11.047157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108fec3-da12-454b-8556-77295476f99c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f108fec3-da12-454b-8556-77295476f99c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:15:11.081568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:11.082479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:15:11.083283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:15:11.083283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:15:11.083283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:11.083283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 13/29] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:11.084244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21483-668a-48c1-a162-4319c6a4ca76 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:11.084349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21483-668a-48c1-a162-4319c6a4ca76 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:11.084588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21483-668a-48c1-a162-4319c6a4ca76 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:11.085886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 14/30] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:11.091342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a201fe9-6675-4fe8-ab91-7739ca487712 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:11.091627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a201fe9-6675-4fe8-ab91-7739ca487712 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:11.091901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a201fe9-6675-4fe8-ab91-7739ca487712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:11.098537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bdfba-d70d-4bc5-b86b-2092a8fd5f46 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:11.129400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7a201fe9-6675-4fe8-ab91-7739ca487712 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:11.130642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7a201fe9-6675-4fe8-ab91-7739ca487712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:11.143167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d159e-be06-4623-b610-d7750d757662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:11.143790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d159e-be06-4623-b610-d7750d757662 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:11.144067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d159e-be06-4623-b610-d7750d757662 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:11.144337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d159e-be06-4623-b610-d7750d757662 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:11.145035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d159e-be06-4623-b610-d7750d757662 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:11.145337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d159e-be06-4623-b610-d7750d757662 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:11.146731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d159e-be06-4623-b610-d7750d757662 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:11.162725 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 17/31] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:11.163813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d159e-be06-4623-b610-d7750d757662 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:11.227622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 15/32] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 30 19:15:11.228517 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a201fe9-6675-4fe8-ab91-7739ca487712 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:11.294152 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:11.357477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d159e-be06-4623-b610-d7750d757662 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:11.388152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:11.404503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47e31c-6151-4a8d-bafc-670bd019295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:11.405138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47e31c-6151-4a8d-bafc-670bd019295f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:11.405514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47e31c-6151-4a8d-bafc-670bd019295f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:11.405791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47e31c-6151-4a8d-bafc-670bd019295f None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:11.406546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47e31c-6151-4a8d-bafc-670bd019295f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:11.406852 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47e31c-6151-4a8d-bafc-670bd019295f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:11.408231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47e31c-6151-4a8d-bafc-670bd019295f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:11.409122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a201fe9-6675-4fe8-ab91-7739ca487712 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:11.426856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 18/33] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:11.427670 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47e31c-6151-4a8d-bafc-670bd019295f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:11.474269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb64dc1-fa44-4538-9752-f6cf54b1161c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:11.474917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb64dc1-fa44-4538-9752-f6cf54b1161c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:11.475193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb64dc1-fa44-4538-9752-f6cf54b1161c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:11.475463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb64dc1-fa44-4538-9752-f6cf54b1161c None admin] PATH_INFO: `/v3/users/02c8dc74a8e74fac884b7eaf15083780` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:11.477297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb64dc1-fa44-4538-9752-f6cf54b1161c None admin] RBAC: Authorizing `identity:get_user(user_id=02c8dc74a8e74fac884b7eaf15083780)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:11.478923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb64dc1-fa44-4538-9752-f6cf54b1161c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:11.481759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 16/34] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:11 2022] GET /identity/v3/users/02c8dc74a8e74fac884b7eaf15083780 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:11.482591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb64dc1-fa44-4538-9752-f6cf54b1161c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:11.563084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47e31c-6151-4a8d-bafc-670bd019295f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:11.603043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:11.613840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b6954-d920-46f7-b422-1ea10f6ac9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:11.614471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b6954-d920-46f7-b422-1ea10f6ac9d6 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:11.614756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b6954-d920-46f7-b422-1ea10f6ac9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:11.615493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b6954-d920-46f7-b422-1ea10f6ac9d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:11.616778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b6954-d920-46f7-b422-1ea10f6ac9d6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:11.618885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b6954-d920-46f7-b422-1ea10f6ac9d6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:11.639808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb64dc1-fa44-4538-9752-f6cf54b1161c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:11.674860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1c11b-eb80-4e6a-bc95-e62ba6c58c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:11.675469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1c11b-eb80-4e6a-bc95-e62ba6c58c59 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:11.675743 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1c11b-eb80-4e6a-bc95-e62ba6c58c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:11.676009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1c11b-eb80-4e6a-bc95-e62ba6c58c59 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:11.676678 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1c11b-eb80-4e6a-bc95-e62ba6c58c59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:11.678089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1c11b-eb80-4e6a-bc95-e62ba6c58c59 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:11.748530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 19/35] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 30 19:15:11 2022] POST /identity/v3/roles => generated 219 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:11.749269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b6954-d920-46f7-b422-1ea10f6ac9d6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:11.824837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 17/36] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:15:11 2022] POST /identity/v3/projects => generated 301 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:11.826208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1c11b-eb80-4e6a-bc95-e62ba6c58c59 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:11.851927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:11.889171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b6954-d920-46f7-b422-1ea10f6ac9d6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:11.947035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e667c-f33e-4932-bd2c-c77e8c14448d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:11.948248 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e667c-f33e-4932-bd2c-c77e8c14448d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:11.948559 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e667c-f33e-4932-bd2c-c77e8c14448d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:11.948834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e667c-f33e-4932-bd2c-c77e8c14448d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:11.950590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e667c-f33e-4932-bd2c-c77e8c14448d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:11.952473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e667c-f33e-4932-bd2c-c77e8c14448d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:11.954362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 20/37] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:11.960875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddb6c8-2202-4b7d-bbc3-d4462c52d5e8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:11.961444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddb6c8-2202-4b7d-bbc3-d4462c52d5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:11.961948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddb6c8-2202-4b7d-bbc3-d4462c52d5e8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:11.963762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 21/38] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:11.966457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddb6c8-2202-4b7d-bbc3-d4462c52d5e8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:12.014178 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1c11b-eb80-4e6a-bc95-e62ba6c58c59 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:12.066558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f27f6a-0172-4756-af3b-f0ca1b40e4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:12.067216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f27f6a-0172-4756-af3b-f0ca1b40e4be None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:12.067499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f27f6a-0172-4756-af3b-f0ca1b40e4be None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:12.067773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f27f6a-0172-4756-af3b-f0ca1b40e4be None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:12.068647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f27f6a-0172-4756-af3b-f0ca1b40e4be None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:12.070210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f27f6a-0172-4756-af3b-f0ca1b40e4be None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:12.093732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 18/39] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 30 19:15:11 2022] POST /identity/v3/roles => generated 225 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:12.095536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef98295-a536-4b35-88f7-3be83690ec50 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:12.096074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef98295-a536-4b35-88f7-3be83690ec50 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:12.097626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef98295-a536-4b35-88f7-3be83690ec50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:12.127684 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0ef98295-a536-4b35-88f7-3be83690ec50 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:12.129471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0ef98295-a536-4b35-88f7-3be83690ec50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:12.171597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:12.223484 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:12.232662 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 19/40] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:12.234090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919fe88-a040-40eb-bfab-d29d4f0d4834 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:12.234372 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919fe88-a040-40eb-bfab-d29d4f0d4834 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:12.234643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919fe88-a040-40eb-bfab-d29d4f0d4834 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:12.236084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 20/41] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:12.241426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b24d0-d9ac-42b4-87f9-52691d5a1954 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:12.242238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b24d0-d9ac-42b4-87f9-52691d5a1954 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:12.242610 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b24d0-d9ac-42b4-87f9-52691d5a1954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:12.245448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddb6c8-2202-4b7d-bbc3-d4462c52d5e8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:12.266372 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-577b24d0-d9ac-42b4-87f9-52691d5a1954 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:12.267510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-577b24d0-d9ac-42b4-87f9-52691d5a1954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:12.295016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68a15f-155d-4aa6-80f4-3b4c052739c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:12.295698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68a15f-155d-4aa6-80f4-3b4c052739c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:12.295984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68a15f-155d-4aa6-80f4-3b4c052739c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:12.296263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68a15f-155d-4aa6-80f4-3b4c052739c6 None admin] PATH_INFO: `/v3/domains/default/users/02c8dc74a8e74fac884b7eaf15083780/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:12.298419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68a15f-155d-4aa6-80f4-3b4c052739c6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=02c8dc74a8e74fac884b7eaf15083780, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:12.299890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68a15f-155d-4aa6-80f4-3b4c052739c6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:12.320428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 22/42] 10.222.0.17 () {58 vars in 1361 bytes} [Fri Sep 30 19:15:11 2022] PUT /identity/v3/domains/default/users/02c8dc74a8e74fac884b7eaf15083780/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:15:12.365566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 21/43] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:12.511024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68a15f-155d-4aa6-80f4-3b4c052739c6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:12.612344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68a15f-155d-4aa6-80f4-3b4c052739c6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:12.627325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:12.639990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b24d0-d9ac-42b4-87f9-52691d5a1954 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:12.653654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5e286-8710-483b-969c-faae680e39fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:12.654303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5e286-8710-483b-969c-faae680e39fb None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:12.654610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5e286-8710-483b-969c-faae680e39fb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:12.654887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5e286-8710-483b-969c-faae680e39fb None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:12.655578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5e286-8710-483b-969c-faae680e39fb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:12.657012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5e286-8710-483b-969c-faae680e39fb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:12.677192 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 23/44] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 30 19:15:12 2022] POST /identity/v3/roles => generated 223 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:12.753155 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b24d0-d9ac-42b4-87f9-52691d5a1954 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:12.782106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:12.791803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea1b4b-3998-420e-af0c-57db3c4d76f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:12.792521 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea1b4b-3998-420e-af0c-57db3c4d76f2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:12.792829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea1b4b-3998-420e-af0c-57db3c4d76f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:12.793110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea1b4b-3998-420e-af0c-57db3c4d76f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:12.795486 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea1b4b-3998-420e-af0c-57db3c4d76f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:12.796146 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea1b4b-3998-420e-af0c-57db3c4d76f2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:12.798065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 22/45] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:12.803683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5e286-8710-483b-969c-faae680e39fb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:12.917194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5e286-8710-483b-969c-faae680e39fb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:12.948352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567e130-3466-4125-8da3-f6f0f4f54e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:12.948993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567e130-3466-4125-8da3-f6f0f4f54e9b None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:12.949274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567e130-3466-4125-8da3-f6f0f4f54e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:12.949544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567e130-3466-4125-8da3-f6f0f4f54e9b None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:12.950201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567e130-3466-4125-8da3-f6f0f4f54e9b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:12.951684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567e130-3466-4125-8da3-f6f0f4f54e9b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:12.980464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 24/46] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:15:12 2022] POST /identity/v3/projects => generated 312 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:15:14.456200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67537cb-7685-45a3-b793-c5d36c906244 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:14.456735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67537cb-7685-45a3-b793-c5d36c906244 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:14.456832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67537cb-7685-45a3-b793-c5d36c906244 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:14.458130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 23/47] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:14.463753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f062-a503-46be-97f0-5b44bf30f6fd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:14.464155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f062-a503-46be-97f0-5b44bf30f6fd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:14.464303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8f062-a503-46be-97f0-5b44bf30f6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:14.483663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-eff8f062-a503-46be-97f0-5b44bf30f6fd None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:14.485180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eff8f062-a503-46be-97f0-5b44bf30f6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:14.504410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:14.551047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 25/48] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:15:14.814215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67537cb-7685-45a3-b793-c5d36c906244 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:14.912442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67537cb-7685-45a3-b793-c5d36c906244 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:14.945685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc9dda-3e99-453f-a506-26e7d3911c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:14.946308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc9dda-3e99-453f-a506-26e7d3911c6b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:14.946584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc9dda-3e99-453f-a506-26e7d3911c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:14.946855 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc9dda-3e99-453f-a506-26e7d3911c6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:14.956485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc9dda-3e99-453f-a506-26e7d3911c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-46cc9dda-3e99-453f-a506-26e7d3911c6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:14.961256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:14.962108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:14.963185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 24/49] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:15:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:14.968080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8f062-a503-46be-97f0-5b44bf30f6fd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:15.073167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8f062-a503-46be-97f0-5b44bf30f6fd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:15.105915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123b658-a038-4202-927b-33684336c617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:15.106519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123b658-a038-4202-927b-33684336c617 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:15.106796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123b658-a038-4202-927b-33684336c617 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:15.107063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123b658-a038-4202-927b-33684336c617 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:15.107725 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123b658-a038-4202-927b-33684336c617 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:15.108023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123b658-a038-4202-927b-33684336c617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:15.109438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123b658-a038-4202-927b-33684336c617 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:15.112424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34885d3-6d59-4fa8-93a4-b1d2e08d32fe None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:15.112713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34885d3-6d59-4fa8-93a4-b1d2e08d32fe None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:15.112991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34885d3-6d59-4fa8-93a4-b1d2e08d32fe None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:15.114320 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 25/50] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:15:15.121042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 26/51] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:15.121166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7a5bb-890c-4dc2-a206-c9eae246b286 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:15.121166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7a5bb-890c-4dc2-a206-c9eae246b286 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:15.121166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7a5bb-890c-4dc2-a206-c9eae246b286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:15.123854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123b658-a038-4202-927b-33684336c617 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:15.138249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-68e7a5bb-890c-4dc2-a206-c9eae246b286 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:15.139416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-68e7a5bb-890c-4dc2-a206-c9eae246b286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:15.143644 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:15.190327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:15.206841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 26/52] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:15.224703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123b658-a038-4202-927b-33684336c617 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:15.256872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886730c6-5a8a-4288-bd9f-7ec386e34d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:15.257516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886730c6-5a8a-4288-bd9f-7ec386e34d56 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:15.257794 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886730c6-5a8a-4288-bd9f-7ec386e34d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:15.258060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886730c6-5a8a-4288-bd9f-7ec386e34d56 None admin] PATH_INFO: `/v3/users/02c8dc74a8e74fac884b7eaf15083780` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:15.259171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886730c6-5a8a-4288-bd9f-7ec386e34d56 None admin] RBAC: Authorizing `identity:get_user(user_id=02c8dc74a8e74fac884b7eaf15083780)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:15.261230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886730c6-5a8a-4288-bd9f-7ec386e34d56 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:15.262776 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 27/53] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:15 2022] GET /identity/v3/users/02c8dc74a8e74fac884b7eaf15083780 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:15.268163 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7a5bb-890c-4dc2-a206-c9eae246b286 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:15.363890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7a5bb-890c-4dc2-a206-c9eae246b286 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:15.399109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44d0bf-2c79-42f7-8d3f-49cfcdb7d7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:15.399725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44d0bf-2c79-42f7-8d3f-49cfcdb7d7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:15.400004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44d0bf-2c79-42f7-8d3f-49cfcdb7d7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:15.400276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44d0bf-2c79-42f7-8d3f-49cfcdb7d7c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:15.402502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44d0bf-2c79-42f7-8d3f-49cfcdb7d7c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:15.403940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44d0bf-2c79-42f7-8d3f-49cfcdb7d7c2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:15.405508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 27/54] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:15.411217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886730c6-5a8a-4288-bd9f-7ec386e34d56 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:15.470231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44d0bf-2c79-42f7-8d3f-49cfcdb7d7c2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:15.494043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:15.509299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886730c6-5a8a-4288-bd9f-7ec386e34d56 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:15.523962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:15.543348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e841f7-b624-4b78-97d2-8870676f594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:15.543946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e841f7-b624-4b78-97d2-8870676f594c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:15.544225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e841f7-b624-4b78-97d2-8870676f594c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:15.544529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e841f7-b624-4b78-97d2-8870676f594c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:15.545251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e841f7-b624-4b78-97d2-8870676f594c None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.domain.id=default, user.id=02c8dc74a8e74fac884b7eaf15083780) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:15.545556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e841f7-b624-4b78-97d2-8870676f594c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:15.547013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e841f7-b624-4b78-97d2-8870676f594c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:15.553015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 28/55] 10.222.0.17 () {56 vars in 1322 bytes} [Fri Sep 30 19:15:15 2022] GET /identity/v3/role_assignments?user.id=02c8dc74a8e74fac884b7eaf15083780&scope.domain.id=default&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:15.568933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44d0bf-2c79-42f7-8d3f-49cfcdb7d7c2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:15.710853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef1359-ab13-4fb3-a20b-fdda5a2c4027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:15.711504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef1359-ab13-4fb3-a20b-fdda5a2c4027 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:15.711781 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef1359-ab13-4fb3-a20b-fdda5a2c4027 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:15.712052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef1359-ab13-4fb3-a20b-fdda5a2c4027 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:15.713254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef1359-ab13-4fb3-a20b-fdda5a2c4027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:15.714711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef1359-ab13-4fb3-a20b-fdda5a2c4027 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:15.716297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 28/56] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:15.722143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e841f7-b624-4b78-97d2-8870676f594c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:15.821658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e841f7-b624-4b78-97d2-8870676f594c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:15.854237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6674491-ef98-4247-ab93-aaae18e7b70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:15.854833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6674491-ef98-4247-ab93-aaae18e7b70f None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:15.855111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6674491-ef98-4247-ab93-aaae18e7b70f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:15.855380 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6674491-ef98-4247-ab93-aaae18e7b70f None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:15.856030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6674491-ef98-4247-ab93-aaae18e7b70f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:15.857499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6674491-ef98-4247-ab93-aaae18e7b70f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:15.882258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 29/57] 10.222.0.17 () {60 vars in 1140 bytes} [Fri Sep 30 19:15:15 2022] POST /identity/v3/projects => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:17.991751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42f61b-f472-4a22-9244-6fab0202cdf1 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:17.992266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42f61b-f472-4a22-9244-6fab0202cdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:17.992266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42f61b-f472-4a22-9244-6fab0202cdf1 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:17.993653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 29/58] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:17.999293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0844b-a48c-488f-b839-423ba348556b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:17.999704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0844b-a48c-488f-b839-423ba348556b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:17.999844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0844b-a48c-488f-b839-423ba348556b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:18.006015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:18.018973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-58f0844b-a48c-488f-b839-423ba348556b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:18.020151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-58f0844b-a48c-488f-b839-423ba348556b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:18.087039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 30/59] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:18.355422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42f61b-f472-4a22-9244-6fab0202cdf1 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:18.453817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42f61b-f472-4a22-9244-6fab0202cdf1 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:18.478612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:18.489696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b7616-ed4c-4d57-897c-51eb3f840d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:18.490313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7616-ed4c-4d57-897c-51eb3f840d3f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:18.490590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7616-ed4c-4d57-897c-51eb3f840d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:18.490860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7616-ed4c-4d57-897c-51eb3f840d3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:18.491949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b7616-ed4c-4d57-897c-51eb3f840d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:18.493412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b7616-ed4c-4d57-897c-51eb3f840d3f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:18.494969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 30/60] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:18.500502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0844b-a48c-488f-b839-423ba348556b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:18.594258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0844b-a48c-488f-b839-423ba348556b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:18.621903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:18.626721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139963d2-9ad5-46c8-bf8e-d8883b266cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:18.627316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139963d2-9ad5-46c8-bf8e-d8883b266cb3 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:18.627593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139963d2-9ad5-46c8-bf8e-d8883b266cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:18.627864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139963d2-9ad5-46c8-bf8e-d8883b266cb3 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:18.628531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139963d2-9ad5-46c8-bf8e-d8883b266cb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:18.629972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139963d2-9ad5-46c8-bf8e-d8883b266cb3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:18.655944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 31/61] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:15:18 2022] POST /identity/v3/users => generated 275 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:21.954616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20758e-3366-4499-9606-f2068ca8632f None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:21.955632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20758e-3366-4499-9606-f2068ca8632f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:21.956189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20758e-3366-4499-9606-f2068ca8632f None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:21.957801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 31/62] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:21.963996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dd913-e054-4836-9756-919e5002c46b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:21.964859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dd913-e054-4836-9756-919e5002c46b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:21.965360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dd913-e054-4836-9756-919e5002c46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:21.992806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4e9dd913-e054-4836-9756-919e5002c46b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:21.995130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4e9dd913-e054-4836-9756-919e5002c46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:22.094581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 32/63] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:22.106058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b497b-d637-421f-8c7a-ed112b7e3f88 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.106616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b497b-d637-421f-8c7a-ed112b7e3f88 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.107083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b497b-d637-421f-8c7a-ed112b7e3f88 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.108691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 32/64] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:22.114161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06b862-486e-40c6-8db3-69411b5d9fb1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.114448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06b862-486e-40c6-8db3-69411b5d9fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.114722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06b862-486e-40c6-8db3-69411b5d9fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.137040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-db06b862-486e-40c6-8db3-69411b5d9fb1 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:22.138233 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-db06b862-486e-40c6-8db3-69411b5d9fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:22.234906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 33/65] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:22.374705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b497b-d637-421f-8c7a-ed112b7e3f88 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:22.419863 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aedcc-151a-4e9e-bb46-8cfd12fbc6e1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.420176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aedcc-151a-4e9e-bb46-8cfd12fbc6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.420439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aedcc-151a-4e9e-bb46-8cfd12fbc6e1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.422783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 34/66] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:15:22.427859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8ac0d-c851-46e2-94c6-c2d06717b781 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.428143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8ac0d-c851-46e2-94c6-c2d06717b781 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.428586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8ac0d-c851-46e2-94c6-c2d06717b781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.453544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a1f8ac0d-c851-46e2-94c6-c2d06717b781 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:22.454735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a1f8ac0d-c851-46e2-94c6-c2d06717b781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:22.508989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b497b-d637-421f-8c7a-ed112b7e3f88 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:22.513385 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:22.545585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 35/67] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:15:22.549734 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fde735-bdb9-413c-93c2-e628a105d2e3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.550181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fde735-bdb9-413c-93c2-e628a105d2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.550598 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fde735-bdb9-413c-93c2-e628a105d2e3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.551992 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 36/68] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:22.558276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfcfc8-3c79-4a41-b18a-bf5234818d70 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.558747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfcfc8-3c79-4a41-b18a-bf5234818d70 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.559161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfcfc8-3c79-4a41-b18a-bf5234818d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.573526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb3cce-a4d4-4177-9177-56d90ba16bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:22.574180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb3cce-a4d4-4177-9177-56d90ba16bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.574617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb3cce-a4d4-4177-9177-56d90ba16bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.575057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb3cce-a4d4-4177-9177-56d90ba16bb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.579134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-25bfcfc8-3c79-4a41-b18a-bf5234818d70 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:22.580526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-25bfcfc8-3c79-4a41-b18a-bf5234818d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:22.597348 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb3cce-a4d4-4177-9177-56d90ba16bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-0ffb3cce-a4d4-4177-9177-56d90ba16bb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:22.602415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:22.603506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:22.604520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:22.606448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 33/69] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:22.607405 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb3cce-a4d4-4177-9177-56d90ba16bb3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:22.657987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:22.663413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 37/70] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:22.664105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfcfc8-3c79-4a41-b18a-bf5234818d70 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:22.728607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb3cce-a4d4-4177-9177-56d90ba16bb3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:22.767516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bef04-5a66-4097-9fd7-742cf039d68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:22.768076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bef04-5a66-4097-9fd7-742cf039d68d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.768356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bef04-5a66-4097-9fd7-742cf039d68d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.768671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bef04-5a66-4097-9fd7-742cf039d68d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.772911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfcfc8-3c79-4a41-b18a-bf5234818d70 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:22.779991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bef04-5a66-4097-9fd7-742cf039d68d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:22.783132 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-369bef04-5a66-4097-9fd7-742cf039d68d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:22.786363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:22.787491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:22.788283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 34/71] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:15:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:22.789051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ddc85-73b0-4da2-bd4b-532221511eca None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.789905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ddc85-73b0-4da2-bd4b-532221511eca None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.789905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ddc85-73b0-4da2-bd4b-532221511eca None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.790678 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 35/72] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:22.798926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a0b5f-e2a3-445f-896c-c6b48a693baf None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.798926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a0b5f-e2a3-445f-896c-c6b48a693baf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.798926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a0b5f-e2a3-445f-896c-c6b48a693baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.818083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be19f08-1107-4a37-bb4a-a9b438c5178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:22.818349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-068a0b5f-e2a3-445f-896c-c6b48a693baf None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:22.818695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be19f08-1107-4a37-bb4a-a9b438c5178e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.818973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be19f08-1107-4a37-bb4a-a9b438c5178e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.819240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be19f08-1107-4a37-bb4a-a9b438c5178e None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.819512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-068a0b5f-e2a3-445f-896c-c6b48a693baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:22.819927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be19f08-1107-4a37-bb4a-a9b438c5178e None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:22.820199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be19f08-1107-4a37-bb4a-a9b438c5178e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:22.821687 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be19f08-1107-4a37-bb4a-a9b438c5178e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:22.830008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 38/73] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:22.835378 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be19f08-1107-4a37-bb4a-a9b438c5178e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:22.893844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 36/74] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:22.897970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a0b5f-e2a3-445f-896c-c6b48a693baf None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:22.939827 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be19f08-1107-4a37-bb4a-a9b438c5178e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:22.949854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:22.980626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50f4f4-1cb8-40a9-9b94-cdcbab2b4ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:22.981235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50f4f4-1cb8-40a9-9b94-cdcbab2b4ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:22.981510 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50f4f4-1cb8-40a9-9b94-cdcbab2b4ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:22.981775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50f4f4-1cb8-40a9-9b94-cdcbab2b4ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:22.982435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50f4f4-1cb8-40a9-9b94-cdcbab2b4ed9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:22.982733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50f4f4-1cb8-40a9-9b94-cdcbab2b4ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:22.984090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50f4f4-1cb8-40a9-9b94-cdcbab2b4ed9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:22.992585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 39/75] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:22.993207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50f4f4-1cb8-40a9-9b94-cdcbab2b4ed9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:23.015617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a0b5f-e2a3-445f-896c-c6b48a693baf None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:23.026669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:23.052870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15c87f-8650-433e-87e8-8176dffc4ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:23.053498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15c87f-8650-433e-87e8-8176dffc4ccd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:23.053777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15c87f-8650-433e-87e8-8176dffc4ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:23.054045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15c87f-8650-433e-87e8-8176dffc4ccd None admin] PATH_INFO: `/v3/users/18593a37ac42418aab61de7fa0ece2c8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:23.068932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15c87f-8650-433e-87e8-8176dffc4ccd None admin] RBAC: Authorizing `identity:get_user(user_id=18593a37ac42418aab61de7fa0ece2c8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:23.070383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15c87f-8650-433e-87e8-8176dffc4ccd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:23.072730 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 37/76] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:22 2022] GET /identity/v3/users/18593a37ac42418aab61de7fa0ece2c8 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:23.073379 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15c87f-8650-433e-87e8-8176dffc4ccd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:23.105604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50f4f4-1cb8-40a9-9b94-cdcbab2b4ed9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:23.141355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c3a33-16ce-41e8-87f8-5ccaf46a5d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:23.142262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c3a33-16ce-41e8-87f8-5ccaf46a5d0d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:23.142712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c3a33-16ce-41e8-87f8-5ccaf46a5d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:23.143187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c3a33-16ce-41e8-87f8-5ccaf46a5d0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:23.153809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c3a33-16ce-41e8-87f8-5ccaf46a5d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f80c3a33-16ce-41e8-87f8-5ccaf46a5d0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:23.159701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:23.160567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:23.161485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:23.163587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 40/77] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:23.169747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c3a33-16ce-41e8-87f8-5ccaf46a5d0d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:23.170468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:23.174367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15c87f-8650-433e-87e8-8176dffc4ccd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:23.209615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138033b9-f6f2-4495-8fd0-ed49129c551e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:23.210599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138033b9-f6f2-4495-8fd0-ed49129c551e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:23.210889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138033b9-f6f2-4495-8fd0-ed49129c551e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:23.211167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138033b9-f6f2-4495-8fd0-ed49129c551e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:23.221861 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138033b9-f6f2-4495-8fd0-ed49129c551e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-138033b9-f6f2-4495-8fd0-ed49129c551e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:23.226657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:23.227666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 38/78] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:15:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:23.228709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138033b9-f6f2-4495-8fd0-ed49129c551e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:23.262138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:23.271500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c3a33-16ce-41e8-87f8-5ccaf46a5d0d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:23.306004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b57f32-63f5-4d5d-9bee-b670d007e913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:23.307109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b57f32-63f5-4d5d-9bee-b670d007e913 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:23.307391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b57f32-63f5-4d5d-9bee-b670d007e913 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:23.307701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b57f32-63f5-4d5d-9bee-b670d007e913 None admin] PATH_INFO: `/v3/users/02c8dc74a8e74fac884b7eaf15083780` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:23.308854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b57f32-63f5-4d5d-9bee-b670d007e913 None admin] RBAC: Authorizing `identity:get_user(user_id=02c8dc74a8e74fac884b7eaf15083780)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:23.310320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b57f32-63f5-4d5d-9bee-b670d007e913 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:23.311890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 41/79] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:23 2022] GET /identity/v3/users/02c8dc74a8e74fac884b7eaf15083780 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:23.312673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b57f32-63f5-4d5d-9bee-b670d007e913 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:23.329068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138033b9-f6f2-4495-8fd0-ed49129c551e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:23.361341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27a45-b34d-4ed8-b151-71b4f7ea4d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:23.361943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27a45-b34d-4ed8-b151-71b4f7ea4d85 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:23.362229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27a45-b34d-4ed8-b151-71b4f7ea4d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:23.362500 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27a45-b34d-4ed8-b151-71b4f7ea4d85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:23.389466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:23.409561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b57f32-63f5-4d5d-9bee-b670d007e913 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:23.432545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc27a45-b34d-4ed8-b151-71b4f7ea4d85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:23.434017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc27a45-b34d-4ed8-b151-71b4f7ea4d85 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:23.442353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a8e1a-c8e5-4e57-a66c-b3cb1308cd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:23.442976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a8e1a-c8e5-4e57-a66c-b3cb1308cd7a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:23.443262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a8e1a-c8e5-4e57-a66c-b3cb1308cd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:23.443532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a8e1a-c8e5-4e57-a66c-b3cb1308cd7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:23.444658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a8e1a-c8e5-4e57-a66c-b3cb1308cd7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:23.446101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a8e1a-c8e5-4e57-a66c-b3cb1308cd7a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:23.447718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 42/80] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:23.448581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a8e1a-c8e5-4e57-a66c-b3cb1308cd7a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:23.476390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:23.532148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 39/81] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:23.537554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27a45-b34d-4ed8-b151-71b4f7ea4d85 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:23.547193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a8e1a-c8e5-4e57-a66c-b3cb1308cd7a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:23.580950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d4d98-00ba-43b1-bc46-0717edd80172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:23.581773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d4d98-00ba-43b1-bc46-0717edd80172 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:23.582057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d4d98-00ba-43b1-bc46-0717edd80172 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:23.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d4d98-00ba-43b1-bc46-0717edd80172 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:23.582988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d4d98-00ba-43b1-bc46-0717edd80172 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:23.583288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d4d98-00ba-43b1-bc46-0717edd80172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:23.584708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d4d98-00ba-43b1-bc46-0717edd80172 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:23.592976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 43/82] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:23.593517 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:23.593818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d4d98-00ba-43b1-bc46-0717edd80172 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:23.638299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27a45-b34d-4ed8-b151-71b4f7ea4d85 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:23.672613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00153f10-3d27-4c6f-a672-33b6ab6ac9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:23.673586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00153f10-3d27-4c6f-a672-33b6ab6ac9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:23.673876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00153f10-3d27-4c6f-a672-33b6ab6ac9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:23.674152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00153f10-3d27-4c6f-a672-33b6ab6ac9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:23.674806 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00153f10-3d27-4c6f-a672-33b6ab6ac9b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:23.675110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00153f10-3d27-4c6f-a672-33b6ab6ac9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:23.676518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00153f10-3d27-4c6f-a672-33b6ab6ac9b8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:23.681964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:23.688433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 40/83] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:15:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:23.689207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00153f10-3d27-4c6f-a672-33b6ab6ac9b8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:23.692286 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d4d98-00ba-43b1-bc46-0717edd80172 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:23.727717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca7f80-6684-4795-b18f-ebf1795ca752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:23.728900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca7f80-6684-4795-b18f-ebf1795ca752 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:23.729185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca7f80-6684-4795-b18f-ebf1795ca752 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:23.729457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca7f80-6684-4795-b18f-ebf1795ca752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:23.788163 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00153f10-3d27-4c6f-a672-33b6ab6ac9b8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:23.794748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca7f80-6684-4795-b18f-ebf1795ca752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:23.796216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca7f80-6684-4795-b18f-ebf1795ca752 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:23.807670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:23.821574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75631b-bd85-4feb-99c1-54d9f4eb3cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:23.822761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75631b-bd85-4feb-99c1-54d9f4eb3cd3 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:23.823047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75631b-bd85-4feb-99c1-54d9f4eb3cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:23.823322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75631b-bd85-4feb-99c1-54d9f4eb3cd3 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:23.823965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75631b-bd85-4feb-99c1-54d9f4eb3cd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:23.825439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75631b-bd85-4feb-99c1-54d9f4eb3cd3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:23.851805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 41/84] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:15:23 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:23.852589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75631b-bd85-4feb-99c1-54d9f4eb3cd3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:23.886734 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:23.898703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 44/85] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:23.899832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca7f80-6684-4795-b18f-ebf1795ca752 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:23.979425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75631b-bd85-4feb-99c1-54d9f4eb3cd3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:24.017335 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e428e1d-2bf7-4ebd-9dd9-2738602a941f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:24.018028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e428e1d-2bf7-4ebd-9dd9-2738602a941f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:24.018256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e428e1d-2bf7-4ebd-9dd9-2738602a941f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:24.018531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e428e1d-2bf7-4ebd-9dd9-2738602a941f None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:24.019689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e428e1d-2bf7-4ebd-9dd9-2738602a941f None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:24.021197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e428e1d-2bf7-4ebd-9dd9-2738602a941f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:24.022837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 42/86] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:23 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:24.024636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e428e1d-2bf7-4ebd-9dd9-2738602a941f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:24.026302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca7f80-6684-4795-b18f-ebf1795ca752 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:24.069795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249379ca-0be2-4677-b805-c1ec5c3572ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:24.069795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249379ca-0be2-4677-b805-c1ec5c3572ce None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:24.070074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249379ca-0be2-4677-b805-c1ec5c3572ce None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:24.070282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249379ca-0be2-4677-b805-c1ec5c3572ce None admin] PATH_INFO: `/v3/users/18593a37ac42418aab61de7fa0ece2c8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:24.071312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249379ca-0be2-4677-b805-c1ec5c3572ce None admin] RBAC: Authorizing `identity:get_user(user_id=18593a37ac42418aab61de7fa0ece2c8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:24.073024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249379ca-0be2-4677-b805-c1ec5c3572ce None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:24.074737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 45/87] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:23 2022] GET /identity/v3/users/18593a37ac42418aab61de7fa0ece2c8 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:24.075394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249379ca-0be2-4677-b805-c1ec5c3572ce None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:24.080358 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:24.142627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e428e1d-2bf7-4ebd-9dd9-2738602a941f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:24.151785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:24.186286 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41b687-0839-4d4d-ad37-5f96e10fa14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:24.186933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41b687-0839-4d4d-ad37-5f96e10fa14b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:24.187207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41b687-0839-4d4d-ad37-5f96e10fa14b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:24.187536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41b687-0839-4d4d-ad37-5f96e10fa14b None admin] PATH_INFO: `/v3/users/18593a37ac42418aab61de7fa0ece2c8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:24.188875 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41b687-0839-4d4d-ad37-5f96e10fa14b None admin] RBAC: Authorizing `identity:get_user(user_id=18593a37ac42418aab61de7fa0ece2c8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:24.190333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41b687-0839-4d4d-ad37-5f96e10fa14b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:24.191921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 43/88] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:24 2022] GET /identity/v3/users/18593a37ac42418aab61de7fa0ece2c8 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:24.192719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41b687-0839-4d4d-ad37-5f96e10fa14b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:24.206820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249379ca-0be2-4677-b805-c1ec5c3572ce None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:24.245884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01827f-551d-4ce8-a63e-6c703946056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:24.246523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01827f-551d-4ce8-a63e-6c703946056c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:24.246807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01827f-551d-4ce8-a63e-6c703946056c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:24.247081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01827f-551d-4ce8-a63e-6c703946056c None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:24.248213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01827f-551d-4ce8-a63e-6c703946056c None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:24.249696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01827f-551d-4ce8-a63e-6c703946056c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:24.251326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 46/89] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:24 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:24.256217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01827f-551d-4ce8-a63e-6c703946056c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:24.321516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41b687-0839-4d4d-ad37-5f96e10fa14b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:24.337525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:24.362176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9d7a6-ffd1-4b87-a5a6-3e803f7e6730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:24.362797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9d7a6-ffd1-4b87-a5a6-3e803f7e6730 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:24.363202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9d7a6-ffd1-4b87-a5a6-3e803f7e6730 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:24.363512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9d7a6-ffd1-4b87-a5a6-3e803f7e6730 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:24.364333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9d7a6-ffd1-4b87-a5a6-3e803f7e6730 None admin] RBAC: Adding query filter params (role.id=43452aaa2c9c4505b4bd3dd1b2412c65, scope.project.id=63a660324ea34aa886a24836744d158b, user.id=18593a37ac42418aab61de7fa0ece2c8) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:24.364665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9d7a6-ffd1-4b87-a5a6-3e803f7e6730 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:24.366142 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9d7a6-ffd1-4b87-a5a6-3e803f7e6730 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:24.372124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01827f-551d-4ce8-a63e-6c703946056c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:24.376071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 44/90] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:24 2022] GET /identity/v3/role_assignments?user.id=18593a37ac42418aab61de7fa0ece2c8&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:24.376730 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9d7a6-ffd1-4b87-a5a6-3e803f7e6730 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:24.403839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:24.426370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629dc8f-e70a-40c5-81ea-e3c5b33bfcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:24.427543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629dc8f-e70a-40c5-81ea-e3c5b33bfcdf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:24.428328 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629dc8f-e70a-40c5-81ea-e3c5b33bfcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:24.428835 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629dc8f-e70a-40c5-81ea-e3c5b33bfcdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:24.562047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9d7a6-ffd1-4b87-a5a6-3e803f7e6730 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:24.578767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629dc8f-e70a-40c5-81ea-e3c5b33bfcdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:24.580286 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629dc8f-e70a-40c5-81ea-e3c5b33bfcdf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:24.606823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b386960-659b-41f3-a8eb-e3befda25550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:24.607458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b386960-659b-41f3-a8eb-e3befda25550 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:24.607739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b386960-659b-41f3-a8eb-e3befda25550 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:24.608014 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b386960-659b-41f3-a8eb-e3befda25550 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:24.688771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:24.725460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b386960-659b-41f3-a8eb-e3befda25550 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:24.726952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b386960-659b-41f3-a8eb-e3befda25550 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:24.803093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 47/91] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:24.803785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629dc8f-e70a-40c5-81ea-e3c5b33bfcdf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:24.833997 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 45/92] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:24.835288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b386960-659b-41f3-a8eb-e3befda25550 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:24.885080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:24.960849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:24.973325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b386960-659b-41f3-a8eb-e3befda25550 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:25.006781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629dc8f-e70a-40c5-81ea-e3c5b33bfcdf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:25.011376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c694c8-d1d4-463b-8177-67adcbe0c4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:25.012032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c694c8-d1d4-463b-8177-67adcbe0c4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:25.012319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c694c8-d1d4-463b-8177-67adcbe0c4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:25.012649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c694c8-d1d4-463b-8177-67adcbe0c4d9 None admin] PATH_INFO: `/v3/projects/a3396cb83fd94b628b0a5bc26b5c50c7` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:25.013790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c694c8-d1d4-463b-8177-67adcbe0c4d9 None admin] RBAC: Authorizing `identity:get_project(project_id=a3396cb83fd94b628b0a5bc26b5c50c7)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:25.015253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c694c8-d1d4-463b-8177-67adcbe0c4d9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:25.017233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 46/93] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:24 2022] GET /identity/v3/projects/a3396cb83fd94b628b0a5bc26b5c50c7 => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:25.021288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c694c8-d1d4-463b-8177-67adcbe0c4d9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:25.043470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cb4b4-601e-4def-ade1-607afeebb087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:25.044136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cb4b4-601e-4def-ade1-607afeebb087 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:25.044450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cb4b4-601e-4def-ade1-607afeebb087 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:25.044783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cb4b4-601e-4def-ade1-607afeebb087 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:25.045567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cb4b4-601e-4def-ade1-607afeebb087 None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=63a660324ea34aa886a24836744d158b, user.id=02c8dc74a8e74fac884b7eaf15083780) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:25.045874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cb4b4-601e-4def-ade1-607afeebb087 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:25.047329 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cb4b4-601e-4def-ade1-607afeebb087 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:25.053462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 48/94] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:24 2022] GET /identity/v3/role_assignments?user.id=02c8dc74a8e74fac884b7eaf15083780&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:25.054017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cb4b4-601e-4def-ade1-607afeebb087 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:25.130084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c694c8-d1d4-463b-8177-67adcbe0c4d9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:25.168316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980136b-455c-48f5-b12e-8a290914a241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:25.169002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980136b-455c-48f5-b12e-8a290914a241 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:25.169285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980136b-455c-48f5-b12e-8a290914a241 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:25.169562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980136b-455c-48f5-b12e-8a290914a241 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:25.170730 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980136b-455c-48f5-b12e-8a290914a241 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:25.172184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980136b-455c-48f5-b12e-8a290914a241 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:25.174174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 47/95] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:25 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:25.195346 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980136b-455c-48f5-b12e-8a290914a241 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:25.201264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:25.213564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cb4b4-601e-4def-ade1-607afeebb087 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:25.224554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:25.251414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62088f3f-0154-4b51-8938-81b3b0dc9c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:25.252034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62088f3f-0154-4b51-8938-81b3b0dc9c53 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:25.252316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62088f3f-0154-4b51-8938-81b3b0dc9c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:25.252623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62088f3f-0154-4b51-8938-81b3b0dc9c53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:25.253369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62088f3f-0154-4b51-8938-81b3b0dc9c53 None admin] RBAC: Adding query filter params (role.id=43452aaa2c9c4505b4bd3dd1b2412c65, scope.project.id=a3396cb83fd94b628b0a5bc26b5c50c7, user.id=18593a37ac42418aab61de7fa0ece2c8) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:25.253679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62088f3f-0154-4b51-8938-81b3b0dc9c53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:25.255256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62088f3f-0154-4b51-8938-81b3b0dc9c53 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:25.262138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 49/96] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:25 2022] GET /identity/v3/role_assignments?user.id=18593a37ac42418aab61de7fa0ece2c8&scope.project.id=a3396cb83fd94b628b0a5bc26b5c50c7&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:25.395985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980136b-455c-48f5-b12e-8a290914a241 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:25.455666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f414af-a38b-4c99-a63a-bf86bcef8556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:25.456302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f414af-a38b-4c99-a63a-bf86bcef8556 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:25.456614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f414af-a38b-4c99-a63a-bf86bcef8556 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:25.456892 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f414af-a38b-4c99-a63a-bf86bcef8556 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:25.457698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f414af-a38b-4c99-a63a-bf86bcef8556 None admin] RBAC: Adding query filter params (role.id=9c17da6cc0e54b67af7cb7fb68638a7e, scope.project.id=63a660324ea34aa886a24836744d158b, user.id=18593a37ac42418aab61de7fa0ece2c8) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:25.458016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f414af-a38b-4c99-a63a-bf86bcef8556 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:25.459512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f414af-a38b-4c99-a63a-bf86bcef8556 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:25.466010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 48/97] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:25 2022] GET /identity/v3/role_assignments?user.id=18593a37ac42418aab61de7fa0ece2c8&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=9c17da6cc0e54b67af7cb7fb68638a7e => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:26.596543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22874c6-282e-4a1f-a8b4-e53492848572 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:26.597039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22874c6-282e-4a1f-a8b4-e53492848572 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:26.597142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22874c6-282e-4a1f-a8b4-e53492848572 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:26.598583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 50/98] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:26.618381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32baad-c786-4dce-8a98-96ef703bd0b7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:26.618890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32baad-c786-4dce-8a98-96ef703bd0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:26.618967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32baad-c786-4dce-8a98-96ef703bd0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:26.641996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2d32baad-c786-4dce-8a98-96ef703bd0b7 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:26.643239 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2d32baad-c786-4dce-8a98-96ef703bd0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:26.734150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 49/99] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:26.786327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd349-534f-4d45-9363-13181425b833 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:26.786623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd349-534f-4d45-9363-13181425b833 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:26.787145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fd349-534f-4d45-9363-13181425b833 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:26.788511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 51/100] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:26.793760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855f146-3dd4-4337-aa8a-7b01aa95b1ee None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:26.794046 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855f146-3dd4-4337-aa8a-7b01aa95b1ee None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:26.794317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855f146-3dd4-4337-aa8a-7b01aa95b1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:26.806629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:26.812956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7855f146-3dd4-4337-aa8a-7b01aa95b1ee None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:26.814141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7855f146-3dd4-4337-aa8a-7b01aa95b1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:26.883012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 50/101] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:27.224485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fd349-534f-4d45-9363-13181425b833 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:27.297773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855f146-3dd4-4337-aa8a-7b01aa95b1ee None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:27.387279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fd349-534f-4d45-9363-13181425b833 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:27.441485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855f146-3dd4-4337-aa8a-7b01aa95b1ee None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:27.450610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65555dab-8d2f-49c9-a65e-3b306a372cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:27.451273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65555dab-8d2f-49c9-a65e-3b306a372cbb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:27.451553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65555dab-8d2f-49c9-a65e-3b306a372cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:27.451826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65555dab-8d2f-49c9-a65e-3b306a372cbb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:27.457209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65555dab-8d2f-49c9-a65e-3b306a372cbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:27.458790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65555dab-8d2f-49c9-a65e-3b306a372cbb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:27.460862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 52/102] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:27.477005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65555dab-8d2f-49c9-a65e-3b306a372cbb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:27.485099 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632331e-f749-4976-bdb5-7f3f69e96b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:27.485743 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632331e-f749-4976-bdb5-7f3f69e96b75 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:27.486027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632331e-f749-4976-bdb5-7f3f69e96b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:27.486306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632331e-f749-4976-bdb5-7f3f69e96b75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:27.496758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632331e-f749-4976-bdb5-7f3f69e96b75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-e632331e-f749-4976-bdb5-7f3f69e96b75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:27.501612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:27.502483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:27.503260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 51/103] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:15:27.504733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:27.514832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632331e-f749-4976-bdb5-7f3f69e96b75 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:27.529439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:27.676829 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65555dab-8d2f-49c9-a65e-3b306a372cbb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:27.712571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632331e-f749-4976-bdb5-7f3f69e96b75 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:27.747285 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee7be2-3358-4988-96c7-dea29c99569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:27.749438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7be2-3358-4988-96c7-dea29c99569c None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:27.749718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7be2-3358-4988-96c7-dea29c99569c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:27.749990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7be2-3358-4988-96c7-dea29c99569c None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:27.751662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee7be2-3358-4988-96c7-dea29c99569c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:27.756129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee7be2-3358-4988-96c7-dea29c99569c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:27.787297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e0ac8-72cd-4614-aaf4-a9aa6d57e17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:27.787910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0ac8-72cd-4614-aaf4-a9aa6d57e17a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:27.788191 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0ac8-72cd-4614-aaf4-a9aa6d57e17a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:27.788497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0ac8-72cd-4614-aaf4-a9aa6d57e17a None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:27.789163 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e0ac8-72cd-4614-aaf4-a9aa6d57e17a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:27.789463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e0ac8-72cd-4614-aaf4-a9aa6d57e17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:27.790906 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e0ac8-72cd-4614-aaf4-a9aa6d57e17a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:27.809735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 52/104] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:27.810988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e760492-5092-4f89-b8f4-4db8c9904f85 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:27.811273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e760492-5092-4f89-b8f4-4db8c9904f85 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:27.811548 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e760492-5092-4f89-b8f4-4db8c9904f85 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:27.813097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 53/105] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:27.818387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 53/106] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:15:27 2022] POST /identity/v3/projects => generated 302 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:27.819169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e760492-5092-4f89-b8f4-4db8c9904f85 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:27.824904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10b3a5-6396-427f-a8e5-110c9cbf4ee1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:27.825199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10b3a5-6396-427f-a8e5-110c9cbf4ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:27.825455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10b3a5-6396-427f-a8e5-110c9cbf4ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:27.861496 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8e10b3a5-6396-427f-a8e5-110c9cbf4ee1 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:27.862718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8e10b3a5-6396-427f-a8e5-110c9cbf4ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:27.934345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:28.010326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 54/107] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 30 19:15:28.023485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e760492-5092-4f89-b8f4-4db8c9904f85 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:28.064930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e0c4a-02ec-47b5-b2c2-8e058d95165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:28.065919 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e0c4a-02ec-47b5-b2c2-8e058d95165b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:28.066417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e0c4a-02ec-47b5-b2c2-8e058d95165b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:28.066886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e0c4a-02ec-47b5-b2c2-8e058d95165b None admin] PATH_INFO: `/v3/users/ddb4a3ba691840c3a8ede64a3460318a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:28.081117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e0c4a-02ec-47b5-b2c2-8e058d95165b None admin] RBAC: Authorizing `identity:get_user(user_id=ddb4a3ba691840c3a8ede64a3460318a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:28.082880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e0c4a-02ec-47b5-b2c2-8e058d95165b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:28.085118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 54/108] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:27 2022] GET /identity/v3/users/ddb4a3ba691840c3a8ede64a3460318a => generated 289 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:28.091018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10b3a5-6396-427f-a8e5-110c9cbf4ee1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:28.097528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:28.240901 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10b3a5-6396-427f-a8e5-110c9cbf4ee1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:28.289004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405f758-a78a-4a74-a157-71bc7e4953ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:28.290404 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405f758-a78a-4a74-a157-71bc7e4953ea None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:28.291645 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405f758-a78a-4a74-a157-71bc7e4953ea None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:28.292329 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405f758-a78a-4a74-a157-71bc7e4953ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:28.356039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af705f3-baaa-4c79-b2bb-98e0cacf11db None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:28.356404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af705f3-baaa-4c79-b2bb-98e0cacf11db None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:28.356603 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af705f3-baaa-4c79-b2bb-98e0cacf11db None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:28.357958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 55/109] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:28.363176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4cbf7-c34d-4332-8fd4-7189ead12cc1 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:28.363461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4cbf7-c34d-4332-8fd4-7189ead12cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:28.363739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4cbf7-c34d-4332-8fd4-7189ead12cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:28.383771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fcf4cbf7-c34d-4332-8fd4-7189ead12cc1 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:28.387604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fcf4cbf7-c34d-4332-8fd4-7189ead12cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:28.415219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405f758-a78a-4a74-a157-71bc7e4953ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:28.419782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405f758-a78a-4a74-a157-71bc7e4953ea None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:28.426166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:28.561003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 56/110] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:15:28.561786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4cbf7-c34d-4332-8fd4-7189ead12cc1 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:28.645308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 55/111] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:28.646643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda3c4d-d669-4698-a744-178ec5916ab4 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:28.646933 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda3c4d-d669-4698-a744-178ec5916ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:28.647211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda3c4d-d669-4698-a744-178ec5916ab4 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:28.648497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 56/112] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:28.653390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda3c4d-d669-4698-a744-178ec5916ab4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:28.656082 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:28.749738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4cbf7-c34d-4332-8fd4-7189ead12cc1 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:28.824316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21a549-e6fe-4c7c-97e9-351173d8b498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:28.824905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21a549-e6fe-4c7c-97e9-351173d8b498 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:28.825183 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21a549-e6fe-4c7c-97e9-351173d8b498 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:28.825451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21a549-e6fe-4c7c-97e9-351173d8b498 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:28.839701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21a549-e6fe-4c7c-97e9-351173d8b498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-9d21a549-e6fe-4c7c-97e9-351173d8b498 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:28.844667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:28.845540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:28.846323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 57/113] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:28.847505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125fdec-f3ca-4d30-801a-68ccbaacb8cc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:28.850637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125fdec-f3ca-4d30-801a-68ccbaacb8cc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:28.850637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125fdec-f3ca-4d30-801a-68ccbaacb8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:28.863602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda3c4d-d669-4698-a744-178ec5916ab4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:28.867580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:28.868311 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a125fdec-f3ca-4d30-801a-68ccbaacb8cc None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:28.869585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a125fdec-f3ca-4d30-801a-68ccbaacb8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:28.930816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab85bf8-e491-460f-b5de-29bd5d74f9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:28.931857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab85bf8-e491-460f-b5de-29bd5d74f9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:28.932137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab85bf8-e491-460f-b5de-29bd5d74f9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:28.932513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab85bf8-e491-460f-b5de-29bd5d74f9b4 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:28.933817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab85bf8-e491-460f-b5de-29bd5d74f9b4 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:28.935266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab85bf8-e491-460f-b5de-29bd5d74f9b4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:28.940531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 57/114] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:28 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:28.940531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab85bf8-e491-460f-b5de-29bd5d74f9b4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:28.944729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 58/115] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:28.945786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125fdec-f3ca-4d30-801a-68ccbaacb8cc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:28.993820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:29.052124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125fdec-f3ca-4d30-801a-68ccbaacb8cc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:29.061411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab85bf8-e491-460f-b5de-29bd5d74f9b4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:29.093696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b9a4c-2e01-4330-94c0-4bd3c5f848fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:29.094264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b9a4c-2e01-4330-94c0-4bd3c5f848fc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.094551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b9a4c-2e01-4330-94c0-4bd3c5f848fc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.094823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b9a4c-2e01-4330-94c0-4bd3c5f848fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.102896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e396eac-be2f-4ad9-984f-c71d8b0ab5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:29.104454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e396eac-be2f-4ad9-984f-c71d8b0ab5d0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.104925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e396eac-be2f-4ad9-984f-c71d8b0ab5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.105336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e396eac-be2f-4ad9-984f-c71d8b0ab5d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.106167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e396eac-be2f-4ad9-984f-c71d8b0ab5d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:29.106615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e396eac-be2f-4ad9-984f-c71d8b0ab5d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:29.107247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b9a4c-2e01-4330-94c0-4bd3c5f848fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:29.108348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e396eac-be2f-4ad9-984f-c71d8b0ab5d0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-5b9b9a4c-2e01-4330-94c0-4bd3c5f848fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:29.112836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:29.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:29.114562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:29.118922 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 59/116] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:29.120343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6ffbf-45c3-4ad8-97fe-fed2b7a5f46f None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.120656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6ffbf-45c3-4ad8-97fe-fed2b7a5f46f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.120928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6ffbf-45c3-4ad8-97fe-fed2b7a5f46f None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.121199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 58/117] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:29.122158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e396eac-be2f-4ad9-984f-c71d8b0ab5d0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:29.122600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 60/118] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:29.126691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6ffbf-45c3-4ad8-97fe-fed2b7a5f46f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:29.168787 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:29.225466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:29.234101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6ffbf-45c3-4ad8-97fe-fed2b7a5f46f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:29.244881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e396eac-be2f-4ad9-984f-c71d8b0ab5d0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:29.268540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4846dfd-4f35-4825-bc80-071f2ddb395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:29.269947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4846dfd-4f35-4825-bc80-071f2ddb395d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.270402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4846dfd-4f35-4825-bc80-071f2ddb395d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.270842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4846dfd-4f35-4825-bc80-071f2ddb395d None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.271709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4846dfd-4f35-4825-bc80-071f2ddb395d None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:29.272143 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4846dfd-4f35-4825-bc80-071f2ddb395d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:29.273694 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4846dfd-4f35-4825-bc80-071f2ddb395d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:29.279738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b0cfd-2164-4709-8833-6047bec797f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:29.280338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0cfd-2164-4709-8833-6047bec797f2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.280648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0cfd-2164-4709-8833-6047bec797f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.280921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0cfd-2164-4709-8833-6047bec797f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.281646 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b0cfd-2164-4709-8833-6047bec797f2 None admin] RBAC: Adding query filter params (role.id=fe9a4b7f28a04337bb3889d5fd2cf17d, scope.project.id=63a660324ea34aa886a24836744d158b, user.id=ddb4a3ba691840c3a8ede64a3460318a) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:29.281957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b0cfd-2164-4709-8833-6047bec797f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:29.283385 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b0cfd-2164-4709-8833-6047bec797f2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:29.284611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 61/119] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:29.285947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960abeb-5018-4592-b6c8-a673acd677ef None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.286233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960abeb-5018-4592-b6c8-a673acd677ef None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.286504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960abeb-5018-4592-b6c8-a673acd677ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.294330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 59/120] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:29 2022] GET /identity/v3/role_assignments?user.id=ddb4a3ba691840c3a8ede64a3460318a&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=fe9a4b7f28a04337bb3889d5fd2cf17d => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:29.301723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b0cfd-2164-4709-8833-6047bec797f2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:29.306003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d960abeb-5018-4592-b6c8-a673acd677ef None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:29.307264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d960abeb-5018-4592-b6c8-a673acd677ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:29.381758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 62/121] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:29.382693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960abeb-5018-4592-b6c8-a673acd677ef None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:29.466991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b0cfd-2164-4709-8833-6047bec797f2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:29.469554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:29.483029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:29.505700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960abeb-5018-4592-b6c8-a673acd677ef None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:29.542428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ac46d-a04c-4a1e-a463-0db6c63038b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:29.543366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ac46d-a04c-4a1e-a463-0db6c63038b9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.543641 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ac46d-a04c-4a1e-a463-0db6c63038b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.543920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ac46d-a04c-4a1e-a463-0db6c63038b9 None admin] PATH_INFO: `/v3/users/18593a37ac42418aab61de7fa0ece2c8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.545373 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ac46d-a04c-4a1e-a463-0db6c63038b9 None admin] RBAC: Authorizing `identity:get_user(user_id=18593a37ac42418aab61de7fa0ece2c8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:29.547585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ac46d-a04c-4a1e-a463-0db6c63038b9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:29.550288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 60/122] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:29 2022] GET /identity/v3/users/18593a37ac42418aab61de7fa0ece2c8 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:29.550997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ac46d-a04c-4a1e-a463-0db6c63038b9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:29.563306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4777-c851-4538-9048-58e02d61e9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:29.563998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4777-c851-4538-9048-58e02d61e9bd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.564346 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4777-c851-4538-9048-58e02d61e9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.564649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4777-c851-4538-9048-58e02d61e9bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.583022 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a4777-c851-4538-9048-58e02d61e9bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ee9a4777-c851-4538-9048-58e02d61e9bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:29.589783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:29.590629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:29.591462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:29.597700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 63/123] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:15:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:29.598504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4777-c851-4538-9048-58e02d61e9bd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:29.726075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4777-c851-4538-9048-58e02d61e9bd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:29.744985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ac46d-a04c-4a1e-a463-0db6c63038b9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:29.767915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0223ae-8b73-44f9-8861-88737792bf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:29.768672 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0223ae-8b73-44f9-8861-88737792bf58 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.768951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0223ae-8b73-44f9-8861-88737792bf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.769320 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0223ae-8b73-44f9-8861-88737792bf58 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.770443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0223ae-8b73-44f9-8861-88737792bf58 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:29.772585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0223ae-8b73-44f9-8861-88737792bf58 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:29.774552 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 64/124] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:29 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:29.775882 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0223ae-8b73-44f9-8861-88737792bf58 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:29.781799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b978c9-255b-4b5f-8585-525a380a00d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:29.782445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b978c9-255b-4b5f-8585-525a380a00d3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.782728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b978c9-255b-4b5f-8585-525a380a00d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.783001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b978c9-255b-4b5f-8585-525a380a00d3 None admin] PATH_INFO: `/v3/users/18593a37ac42418aab61de7fa0ece2c8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.784134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b978c9-255b-4b5f-8585-525a380a00d3 None admin] RBAC: Authorizing `identity:get_user(user_id=18593a37ac42418aab61de7fa0ece2c8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:29.786163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b978c9-255b-4b5f-8585-525a380a00d3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:29.788694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 61/125] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:29 2022] GET /identity/v3/users/18593a37ac42418aab61de7fa0ece2c8 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:29.789769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b978c9-255b-4b5f-8585-525a380a00d3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:29.795871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:29.835502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:29.891755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0223ae-8b73-44f9-8861-88737792bf58 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:29.922243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b978c9-255b-4b5f-8585-525a380a00d3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:29.927664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff5985-43a8-44a0-a30d-14b33a9c1e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:29.928287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff5985-43a8-44a0-a30d-14b33a9c1e3d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.928594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff5985-43a8-44a0-a30d-14b33a9c1e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.928868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff5985-43a8-44a0-a30d-14b33a9c1e3d None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.929527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff5985-43a8-44a0-a30d-14b33a9c1e3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:29.929828 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff5985-43a8-44a0-a30d-14b33a9c1e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:29.931195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff5985-43a8-44a0-a30d-14b33a9c1e3d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:29.940258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 65/126] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:29.941723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff5985-43a8-44a0-a30d-14b33a9c1e3d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:29.965541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d0c66-e2eb-4ad5-8001-3bb759bdeda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:29.966826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d0c66-e2eb-4ad5-8001-3bb759bdeda2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:29.967440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d0c66-e2eb-4ad5-8001-3bb759bdeda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:29.967907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d0c66-e2eb-4ad5-8001-3bb759bdeda2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:29.982272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d0c66-e2eb-4ad5-8001-3bb759bdeda2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0c9d0c66-e2eb-4ad5-8001-3bb759bdeda2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:29.989659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:29.990541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:29.991333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:29.991955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 62/127] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:15:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:29.997045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d0c66-e2eb-4ad5-8001-3bb759bdeda2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:30.033836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:30.054384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff5985-43a8-44a0-a30d-14b33a9c1e3d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:30.092339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93c38b-012f-4da5-a758-b57a3bbfc6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:30.093601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93c38b-012f-4da5-a758-b57a3bbfc6e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:30.093985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93c38b-012f-4da5-a758-b57a3bbfc6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:30.094360 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93c38b-012f-4da5-a758-b57a3bbfc6e3 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b/users/18593a37ac42418aab61de7fa0ece2c8/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:30.097011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93c38b-012f-4da5-a758-b57a3bbfc6e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=63a660324ea34aa886a24836744d158b, user_id=18593a37ac42418aab61de7fa0ece2c8, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:30.098578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93c38b-012f-4da5-a758-b57a3bbfc6e3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:30.113532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:30.118459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 66/128] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:15:29 2022] PUT /identity/v3/projects/63a660324ea34aa886a24836744d158b/users/18593a37ac42418aab61de7fa0ece2c8/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:15:30.119955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93c38b-012f-4da5-a758-b57a3bbfc6e3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:30.124620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d0c66-e2eb-4ad5-8001-3bb759bdeda2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:30.179039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e6384-4e9b-44a4-b833-9b47fa575da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:30.180443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e6384-4e9b-44a4-b833-9b47fa575da5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:30.180971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e6384-4e9b-44a4-b833-9b47fa575da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:30.181976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e6384-4e9b-44a4-b833-9b47fa575da5 None admin] PATH_INFO: `/v3/projects/a3396cb83fd94b628b0a5bc26b5c50c7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:30.183793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e6384-4e9b-44a4-b833-9b47fa575da5 None admin] RBAC: Authorizing `identity:get_project(project_id=a3396cb83fd94b628b0a5bc26b5c50c7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:30.185823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e6384-4e9b-44a4-b833-9b47fa575da5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:30.189052 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 63/129] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:29 2022] GET /identity/v3/projects/a3396cb83fd94b628b0a5bc26b5c50c7 => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:30.190905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e6384-4e9b-44a4-b833-9b47fa575da5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:30.236276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93c38b-012f-4da5-a758-b57a3bbfc6e3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:30.287427 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090f062-06a4-45cb-a487-22fc7fa38f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:30.289201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090f062-06a4-45cb-a487-22fc7fa38f60 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:30.289716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090f062-06a4-45cb-a487-22fc7fa38f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:30.290326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090f062-06a4-45cb-a487-22fc7fa38f60 None admin] PATH_INFO: `/v3/users/02c8dc74a8e74fac884b7eaf15083780` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:30.291992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090f062-06a4-45cb-a487-22fc7fa38f60 None admin] RBAC: Authorizing `identity:get_user(user_id=02c8dc74a8e74fac884b7eaf15083780)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:30.293837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090f062-06a4-45cb-a487-22fc7fa38f60 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:30.296506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 67/130] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:30 2022] GET /identity/v3/users/02c8dc74a8e74fac884b7eaf15083780 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:30.297620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090f062-06a4-45cb-a487-22fc7fa38f60 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:30.309720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:30.365164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e6384-4e9b-44a4-b833-9b47fa575da5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:30.418526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7197051-208a-4a99-b4a7-359b0c96b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:30.419317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7197051-208a-4a99-b4a7-359b0c96b80f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:30.419760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7197051-208a-4a99-b4a7-359b0c96b80f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:30.420046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7197051-208a-4a99-b4a7-359b0c96b80f None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:30.420862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7197051-208a-4a99-b4a7-359b0c96b80f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:30.421173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7197051-208a-4a99-b4a7-359b0c96b80f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:30.422615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7197051-208a-4a99-b4a7-359b0c96b80f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:30.423362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090f062-06a4-45cb-a487-22fc7fa38f60 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:30.433748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 64/131] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:15:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:30.434710 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7197051-208a-4a99-b4a7-359b0c96b80f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:30.459906 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3782c-6f0b-46bb-bbd8-68091fec24c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:30.460711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3782c-6f0b-46bb-bbd8-68091fec24c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:30.460993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3782c-6f0b-46bb-bbd8-68091fec24c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:30.461272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3782c-6f0b-46bb-bbd8-68091fec24c3 None admin] PATH_INFO: `/v3/projects/a3396cb83fd94b628b0a5bc26b5c50c7/users/18593a37ac42418aab61de7fa0ece2c8/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:30.463297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3782c-6f0b-46bb-bbd8-68091fec24c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3396cb83fd94b628b0a5bc26b5c50c7, user_id=18593a37ac42418aab61de7fa0ece2c8, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:30.464924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3782c-6f0b-46bb-bbd8-68091fec24c3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:30.467492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:30.483305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 68/132] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:15:30 2022] PUT /identity/v3/projects/a3396cb83fd94b628b0a5bc26b5c50c7/users/18593a37ac42418aab61de7fa0ece2c8/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:15:30.484336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3782c-6f0b-46bb-bbd8-68091fec24c3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:30.562167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7197051-208a-4a99-b4a7-359b0c96b80f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:30.571088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:30.602438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f3ec6-105b-4ad2-96ca-c79eb483e541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:30.603881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f3ec6-105b-4ad2-96ca-c79eb483e541 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:30.604306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f3ec6-105b-4ad2-96ca-c79eb483e541 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:30.604712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f3ec6-105b-4ad2-96ca-c79eb483e541 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:30.606153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f3ec6-105b-4ad2-96ca-c79eb483e541 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:30.607858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f3ec6-105b-4ad2-96ca-c79eb483e541 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:30.609805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 65/133] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:30 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:30.615887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f3ec6-105b-4ad2-96ca-c79eb483e541 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:30.621523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3782c-6f0b-46bb-bbd8-68091fec24c3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:30.663103 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2489959-8f7d-4018-87fb-bf538c20775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:30.664032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2489959-8f7d-4018-87fb-bf538c20775c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:30.664592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2489959-8f7d-4018-87fb-bf538c20775c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:30.665013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2489959-8f7d-4018-87fb-bf538c20775c None admin] PATH_INFO: `/v3/users/18593a37ac42418aab61de7fa0ece2c8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:30.666917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2489959-8f7d-4018-87fb-bf538c20775c None admin] RBAC: Authorizing `identity:get_user(user_id=18593a37ac42418aab61de7fa0ece2c8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:30.669346 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2489959-8f7d-4018-87fb-bf538c20775c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:30.671387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 69/134] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:30 2022] GET /identity/v3/users/18593a37ac42418aab61de7fa0ece2c8 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:30.679898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2489959-8f7d-4018-87fb-bf538c20775c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:30.748639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:30.765728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f3ec6-105b-4ad2-96ca-c79eb483e541 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:30.803546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2489959-8f7d-4018-87fb-bf538c20775c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:30.834150 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd2234-53fb-43a0-a099-bfc5f0724324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:30.834815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd2234-53fb-43a0-a099-bfc5f0724324 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:30.835221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd2234-53fb-43a0-a099-bfc5f0724324 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:30.835595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd2234-53fb-43a0-a099-bfc5f0724324 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b/users/02c8dc74a8e74fac884b7eaf15083780/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:30.838325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd2234-53fb-43a0-a099-bfc5f0724324 None admin] RBAC: Authorizing `identity:create_grant(project_id=63a660324ea34aa886a24836744d158b, user_id=02c8dc74a8e74fac884b7eaf15083780, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:30.839893 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd2234-53fb-43a0-a099-bfc5f0724324 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:30.843132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9746843-eecd-4493-b7dc-b973efde08e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:30.843891 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9746843-eecd-4493-b7dc-b973efde08e2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:30.844174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9746843-eecd-4493-b7dc-b973efde08e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:30.844490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9746843-eecd-4493-b7dc-b973efde08e2 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:30.846045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9746843-eecd-4493-b7dc-b973efde08e2 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:30.847678 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9746843-eecd-4493-b7dc-b973efde08e2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:30.849641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 70/135] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:30 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:30.851792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c98a5-17ea-4580-9c6c-c3cde690d855 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:30.852048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c98a5-17ea-4580-9c6c-c3cde690d855 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:30.852435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c98a5-17ea-4580-9c6c-c3cde690d855 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:30.857782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 71/136] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:30.858919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 66/137] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:15:30 2022] PUT /identity/v3/projects/63a660324ea34aa886a24836744d158b/users/02c8dc74a8e74fac884b7eaf15083780/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:15:30.862220 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c98a5-17ea-4580-9c6c-c3cde690d855 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:30.862854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3bae4-81cd-472b-8315-39a638c2dcba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:30.863375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3bae4-81cd-472b-8315-39a638c2dcba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:30.863881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3bae4-81cd-472b-8315-39a638c2dcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:30.866115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:30.897912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-acf3bae4-81cd-472b-8315-39a638c2dcba None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:30.899426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-acf3bae4-81cd-472b-8315-39a638c2dcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:31.007422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 67/138] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:31.052649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c98a5-17ea-4580-9c6c-c3cde690d855 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:31.098897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff091842-a917-4ad2-9136-ea9ed05a6562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:31.099519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff091842-a917-4ad2-9136-ea9ed05a6562 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:31.099771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff091842-a917-4ad2-9136-ea9ed05a6562 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:31.100045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff091842-a917-4ad2-9136-ea9ed05a6562 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b/users/18593a37ac42418aab61de7fa0ece2c8/roles/9c17da6cc0e54b67af7cb7fb68638a7e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:31.102263 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff091842-a917-4ad2-9136-ea9ed05a6562 None admin] RBAC: Authorizing `identity:create_grant(project_id=63a660324ea34aa886a24836744d158b, user_id=18593a37ac42418aab61de7fa0ece2c8, role_id=9c17da6cc0e54b67af7cb7fb68638a7e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:31.103793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff091842-a917-4ad2-9136-ea9ed05a6562 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:31.122010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 72/139] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:15:30 2022] PUT /identity/v3/projects/63a660324ea34aa886a24836744d158b/users/18593a37ac42418aab61de7fa0ece2c8/roles/9c17da6cc0e54b67af7cb7fb68638a7e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:15:31.360874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3bae4-81cd-472b-8315-39a638c2dcba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:31.484059 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:31.505079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3bae4-81cd-472b-8315-39a638c2dcba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:31.551150 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db44a88-6133-4fda-a686-2fe19fdcface None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:31.551785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db44a88-6133-4fda-a686-2fe19fdcface None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:31.552115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db44a88-6133-4fda-a686-2fe19fdcface None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:31.552419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db44a88-6133-4fda-a686-2fe19fdcface None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:31.554621 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db44a88-6133-4fda-a686-2fe19fdcface None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:31.556070 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db44a88-6133-4fda-a686-2fe19fdcface None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:31.558300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 68/140] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:31.563612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff091842-a917-4ad2-9136-ea9ed05a6562 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:31.685614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:31.736231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff091842-a917-4ad2-9136-ea9ed05a6562 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:31.775814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4969304-5305-4698-a220-0eae9f372828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:31.776465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4969304-5305-4698-a220-0eae9f372828 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:31.776747 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4969304-5305-4698-a220-0eae9f372828 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:31.777076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4969304-5305-4698-a220-0eae9f372828 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:31.777742 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4969304-5305-4698-a220-0eae9f372828 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:31.779192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4969304-5305-4698-a220-0eae9f372828 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:31.803476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 73/141] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:15:31 2022] POST /identity/v3/users => generated 283 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:32.124345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5bf83-8e40-4303-bf08-e7a4ceff5bd2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:32.124869 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5bf83-8e40-4303-bf08-e7a4ceff5bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:32.124869 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5bf83-8e40-4303-bf08-e7a4ceff5bd2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:32.126169 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 69/142] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:32.142292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfa562-f419-4d9c-b7d9-0e84b97bf1b8 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:32.142829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfa562-f419-4d9c-b7d9-0e84b97bf1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:32.142829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfa562-f419-4d9c-b7d9-0e84b97bf1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:32.166172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6fcfa562-f419-4d9c-b7d9-0e84b97bf1b8 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:32.167354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6fcfa562-f419-4d9c-b7d9-0e84b97bf1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:32.253617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 74/143] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:32.755442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5bf83-8e40-4303-bf08-e7a4ceff5bd2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:32.871275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5bf83-8e40-4303-bf08-e7a4ceff5bd2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:32.905969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe97efc-a84d-471e-83a3-30daea1a4ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:32.906605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe97efc-a84d-471e-83a3-30daea1a4ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:32.906883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe97efc-a84d-471e-83a3-30daea1a4ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:32.907154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe97efc-a84d-471e-83a3-30daea1a4ee1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:32.918538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe97efc-a84d-471e-83a3-30daea1a4ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ebe97efc-a84d-471e-83a3-30daea1a4ee1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:32.923853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:32.924807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:32.925637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 70/144] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:32.933416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfa562-f419-4d9c-b7d9-0e84b97bf1b8 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:32.989966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:33.045956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfa562-f419-4d9c-b7d9-0e84b97bf1b8 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:33.085693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146d169-2352-4403-a62a-df353e3d8f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:33.086320 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146d169-2352-4403-a62a-df353e3d8f4a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:33.086585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146d169-2352-4403-a62a-df353e3d8f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:33.086848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146d169-2352-4403-a62a-df353e3d8f4a None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:33.088048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146d169-2352-4403-a62a-df353e3d8f4a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:33.088348 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146d169-2352-4403-a62a-df353e3d8f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:33.089739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146d169-2352-4403-a62a-df353e3d8f4a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:33.099064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 75/145] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:33.105674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe97efc-a84d-471e-83a3-30daea1a4ee1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:33.123903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:33.222261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe97efc-a84d-471e-83a3-30daea1a4ee1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:33.264764 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0503b3-2015-48ed-aa96-177c95ee7034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:33.265398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0503b3-2015-48ed-aa96-177c95ee7034 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:33.265671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0503b3-2015-48ed-aa96-177c95ee7034 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:33.265994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0503b3-2015-48ed-aa96-177c95ee7034 None admin] PATH_INFO: `/v3/users/ddb4a3ba691840c3a8ede64a3460318a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:33.269377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0503b3-2015-48ed-aa96-177c95ee7034 None admin] RBAC: Authorizing `identity:get_user(user_id=ddb4a3ba691840c3a8ede64a3460318a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:33.270853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0503b3-2015-48ed-aa96-177c95ee7034 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:33.272562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 71/146] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:33 2022] GET /identity/v3/users/ddb4a3ba691840c3a8ede64a3460318a => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:33.278931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146d169-2352-4403-a62a-df353e3d8f4a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:33.406034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146d169-2352-4403-a62a-df353e3d8f4a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:33.419320 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:33.448785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf085495-d6e2-4a2e-8551-3a9c90045369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:33.449792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085495-d6e2-4a2e-8551-3a9c90045369 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:33.450264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085495-d6e2-4a2e-8551-3a9c90045369 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:33.450798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085495-d6e2-4a2e-8551-3a9c90045369 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:33.453124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf085495-d6e2-4a2e-8551-3a9c90045369 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:33.454598 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf085495-d6e2-4a2e-8551-3a9c90045369 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:33.456668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 76/147] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:33 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:33.462120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0503b3-2015-48ed-aa96-177c95ee7034 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:33.573038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:33.600583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0503b3-2015-48ed-aa96-177c95ee7034 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:33.636871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db339bf5-87cd-4379-bf4c-173f3ff3c00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:33.637895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db339bf5-87cd-4379-bf4c-173f3ff3c00f None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:33.638394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db339bf5-87cd-4379-bf4c-173f3ff3c00f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:33.638866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db339bf5-87cd-4379-bf4c-173f3ff3c00f None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b/users/ddb4a3ba691840c3a8ede64a3460318a/roles/fe9a4b7f28a04337bb3889d5fd2cf17d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:33.641048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db339bf5-87cd-4379-bf4c-173f3ff3c00f None admin] RBAC: Authorizing `identity:create_grant(project_id=63a660324ea34aa886a24836744d158b, user_id=ddb4a3ba691840c3a8ede64a3460318a, role_id=fe9a4b7f28a04337bb3889d5fd2cf17d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:33.642756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db339bf5-87cd-4379-bf4c-173f3ff3c00f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:33.664557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 72/148] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:15:33 2022] PUT /identity/v3/projects/63a660324ea34aa886a24836744d158b/users/ddb4a3ba691840c3a8ede64a3460318a/roles/fe9a4b7f28a04337bb3889d5fd2cf17d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:15:33.987393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736abc2-fadc-46be-9b43-2ef9e8e6a764 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:33.987780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736abc2-fadc-46be-9b43-2ef9e8e6a764 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:33.987964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736abc2-fadc-46be-9b43-2ef9e8e6a764 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:33.989355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 77/149] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:33.994851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61daf2-4eb5-4da2-bdb8-9fe33c7fe32a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:33.995411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61daf2-4eb5-4da2-bdb8-9fe33c7fe32a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:33.995884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61daf2-4eb5-4da2-bdb8-9fe33c7fe32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:34.003078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e711d-be77-430e-81a6-168dfba98e33 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:34.003649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e711d-be77-430e-81a6-168dfba98e33 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:34.004111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e711d-be77-430e-81a6-168dfba98e33 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:34.005658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 78/150] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:15:34.010903 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e1abf-c9f4-472a-9d28-0de8dd8247f0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:34.011461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e1abf-c9f4-472a-9d28-0de8dd8247f0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:34.011931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e1abf-c9f4-472a-9d28-0de8dd8247f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:34.035507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5c61daf2-4eb5-4da2-bdb8-9fe33c7fe32a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:34.039017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5c61daf2-4eb5-4da2-bdb8-9fe33c7fe32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:34.045716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f39e1abf-c9f4-472a-9d28-0de8dd8247f0 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:34.047754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f39e1abf-c9f4-472a-9d28-0de8dd8247f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:34.205997 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 79/151] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:15:34.207775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 73/152] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:34.229903 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc752f-5759-4f5e-bdc1-ba875214a5ee None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:34.230224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc752f-5759-4f5e-bdc1-ba875214a5ee None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:34.230452 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc752f-5759-4f5e-bdc1-ba875214a5ee None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:34.231811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 80/153] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:34.237297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d4fb2-3344-45a5-bcd3-b73915d69678 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:34.237817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d4fb2-3344-45a5-bcd3-b73915d69678 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:34.237899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d4fb2-3344-45a5-bcd3-b73915d69678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:34.256880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b07d4fb2-3344-45a5-bcd3-b73915d69678 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:34.258068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b07d4fb2-3344-45a5-bcd3-b73915d69678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:34.326561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:34.335268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 74/154] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:34.522063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ebc9c-4b98-4b65-ab54-10ff5fb07c25 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:34.522565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ebc9c-4b98-4b65-ab54-10ff5fb07c25 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:34.522908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ebc9c-4b98-4b65-ab54-10ff5fb07c25 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:34.525597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 81/155] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:34.539042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ef644-90ad-45f5-941f-ef2241bc71a8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:34.539323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ef644-90ad-45f5-941f-ef2241bc71a8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:34.539597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ef644-90ad-45f5-941f-ef2241bc71a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:34.559276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3a0ef644-90ad-45f5-941f-ef2241bc71a8 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:34.560481 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3a0ef644-90ad-45f5-941f-ef2241bc71a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:34.649668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 75/156] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:34.682568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ebc9c-4b98-4b65-ab54-10ff5fb07c25 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:34.776465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ef644-90ad-45f5-941f-ef2241bc71a8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:34.793354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ebc9c-4b98-4b65-ab54-10ff5fb07c25 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:34.803496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:34.830916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ddbe1-8811-4b7d-ba45-b02870a7ed3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:34.832033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ddbe1-8811-4b7d-ba45-b02870a7ed3e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:34.832544 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ddbe1-8811-4b7d-ba45-b02870a7ed3e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:34.832999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ddbe1-8811-4b7d-ba45-b02870a7ed3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:34.845962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ddbe1-8811-4b7d-ba45-b02870a7ed3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-226ddbe1-8811-4b7d-ba45-b02870a7ed3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:34.851667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:34.852615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:34.853792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:34.856101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 82/157] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:34.862121 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ddbe1-8811-4b7d-ba45-b02870a7ed3e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:34.960279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ef644-90ad-45f5-941f-ef2241bc71a8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:34.985241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ddbe1-8811-4b7d-ba45-b02870a7ed3e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:34.999057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:35.010467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ff332-b611-4f82-b99b-49829773b4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:35.011105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ff332-b611-4f82-b99b-49829773b4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:35.011382 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ff332-b611-4f82-b99b-49829773b4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:35.011668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ff332-b611-4f82-b99b-49829773b4e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:35.033508 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ff332-b611-4f82-b99b-49829773b4e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-247ff332-b611-4f82-b99b-49829773b4e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:35.039181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:35.040295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 76/158] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:35.041738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ff332-b611-4f82-b99b-49829773b4e0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:35.081652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b0a48-b2e0-47b7-9410-409366146b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:35.082270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b0a48-b2e0-47b7-9410-409366146b72 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:35.082546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b0a48-b2e0-47b7-9410-409366146b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:35.082814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b0a48-b2e0-47b7-9410-409366146b72 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:35.083482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b0a48-b2e0-47b7-9410-409366146b72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:35.083784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b0a48-b2e0-47b7-9410-409366146b72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:35.085185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b0a48-b2e0-47b7-9410-409366146b72 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:35.099382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 83/159] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:35.100186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b0a48-b2e0-47b7-9410-409366146b72 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:35.207664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:35.211542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ff332-b611-4f82-b99b-49829773b4e0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:35.283269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bc250-3849-415c-ab96-669c6fc7a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:35.283945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bc250-3849-415c-ab96-669c6fc7a2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:35.284173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bc250-3849-415c-ab96-669c6fc7a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:35.288494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bc250-3849-415c-ab96-669c6fc7a2d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:35.313249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bc250-3849-415c-ab96-669c6fc7a2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fe4bc250-3849-415c-ab96-669c6fc7a2d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:35.318031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:35.318917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:35.319837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:35.321525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 77/160] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:15:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:35.322939 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bc250-3849-415c-ab96-669c6fc7a2d4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:35.342169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b0a48-b2e0-47b7-9410-409366146b72 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:35.408008 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec533d87-a312-474c-9412-c9591eeae050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:35.408676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec533d87-a312-474c-9412-c9591eeae050 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:35.408955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec533d87-a312-474c-9412-c9591eeae050 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:35.409227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec533d87-a312-474c-9412-c9591eeae050 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:35.409910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec533d87-a312-474c-9412-c9591eeae050 None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:35.410208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec533d87-a312-474c-9412-c9591eeae050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:35.411592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec533d87-a312-474c-9412-c9591eeae050 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:35.434272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 84/161] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:35.435142 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec533d87-a312-474c-9412-c9591eeae050 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:35.438497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:35.542425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bc250-3849-415c-ab96-669c6fc7a2d4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:35.557016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:35.574299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec533d87-a312-474c-9412-c9591eeae050 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:35.609930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296b34a-971d-4b62-aead-e2e94081d236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:35.610579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296b34a-971d-4b62-aead-e2e94081d236 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:35.610850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296b34a-971d-4b62-aead-e2e94081d236 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:35.611118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296b34a-971d-4b62-aead-e2e94081d236 None admin] PATH_INFO: `/v3/users/18593a37ac42418aab61de7fa0ece2c8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:35.615883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296b34a-971d-4b62-aead-e2e94081d236 None admin] RBAC: Authorizing `identity:get_user(user_id=18593a37ac42418aab61de7fa0ece2c8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:35.617383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296b34a-971d-4b62-aead-e2e94081d236 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:35.619317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 78/162] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:35 2022] GET /identity/v3/users/18593a37ac42418aab61de7fa0ece2c8 => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:35.620090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296b34a-971d-4b62-aead-e2e94081d236 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:35.664278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b9156-5c3b-49f1-8af3-db5a457c5096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:35.664927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b9156-5c3b-49f1-8af3-db5a457c5096 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:35.665206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b9156-5c3b-49f1-8af3-db5a457c5096 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:35.665477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b9156-5c3b-49f1-8af3-db5a457c5096 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:35.666670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b9156-5c3b-49f1-8af3-db5a457c5096 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:35.666978 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b9156-5c3b-49f1-8af3-db5a457c5096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:35.668382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b9156-5c3b-49f1-8af3-db5a457c5096 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:35.687505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 85/163] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:15:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:35.688150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b9156-5c3b-49f1-8af3-db5a457c5096 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:35.789182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296b34a-971d-4b62-aead-e2e94081d236 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:35.817974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:35.849093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351bbe4-e7b3-4183-9d2c-58c724fcf42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:35.849741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351bbe4-e7b3-4183-9d2c-58c724fcf42d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:35.850040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351bbe4-e7b3-4183-9d2c-58c724fcf42d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:35.850447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351bbe4-e7b3-4183-9d2c-58c724fcf42d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:35.880456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351bbe4-e7b3-4183-9d2c-58c724fcf42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e351bbe4-e7b3-4183-9d2c-58c724fcf42d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:35.887216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:35.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 79/164] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:15:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:35.889212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351bbe4-e7b3-4183-9d2c-58c724fcf42d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:35.901302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b9156-5c3b-49f1-8af3-db5a457c5096 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:35.985751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dbbc5-8a33-4f35-b18a-069676d16856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:35.986771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dbbc5-8a33-4f35-b18a-069676d16856 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:35.987376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dbbc5-8a33-4f35-b18a-069676d16856 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:35.987900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dbbc5-8a33-4f35-b18a-069676d16856 None admin] PATH_INFO: `/v3/users/18593a37ac42418aab61de7fa0ece2c8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:35.989296 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55dbbc5-8a33-4f35-b18a-069676d16856 None admin] RBAC: Authorizing `identity:get_user(user_id=18593a37ac42418aab61de7fa0ece2c8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:35.991094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55dbbc5-8a33-4f35-b18a-069676d16856 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:35.993255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 86/165] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:35 2022] GET /identity/v3/users/18593a37ac42418aab61de7fa0ece2c8 => generated 275 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:35.994405 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dbbc5-8a33-4f35-b18a-069676d16856 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:36.038065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:36.042788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351bbe4-e7b3-4183-9d2c-58c724fcf42d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:36.137154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3595c8b-cddd-4857-b1fc-6ca70d1979ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:36.137697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595c8b-cddd-4857-b1fc-6ca70d1979ac None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:36.137983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595c8b-cddd-4857-b1fc-6ca70d1979ac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:36.138262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595c8b-cddd-4857-b1fc-6ca70d1979ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:36.209128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dbbc5-8a33-4f35-b18a-069676d16856 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:36.231653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:36.277316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3595c8b-cddd-4857-b1fc-6ca70d1979ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:36.278813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3595c8b-cddd-4857-b1fc-6ca70d1979ac None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:36.280642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10efb4-6bec-4e36-bc07-3ef6ca496d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:36.281540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10efb4-6bec-4e36-bc07-3ef6ca496d99 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:36.281963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10efb4-6bec-4e36-bc07-3ef6ca496d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:36.282384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10efb4-6bec-4e36-bc07-3ef6ca496d99 None admin] PATH_INFO: `/v3/users/18593a37ac42418aab61de7fa0ece2c8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:36.283740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10efb4-6bec-4e36-bc07-3ef6ca496d99 None admin] RBAC: Authorizing `identity:get_user(user_id=18593a37ac42418aab61de7fa0ece2c8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:36.285428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10efb4-6bec-4e36-bc07-3ef6ca496d99 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:36.292805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 87/166] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:35 2022] GET /identity/v3/users/18593a37ac42418aab61de7fa0ece2c8 => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:36.297824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10efb4-6bec-4e36-bc07-3ef6ca496d99 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:36.527565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:36.529214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 80/167] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:36.530160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3595c8b-cddd-4857-b1fc-6ca70d1979ac None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:36.547547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10efb4-6bec-4e36-bc07-3ef6ca496d99 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:36.602652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaa321-50af-436d-b771-2db16e18040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:36.604552 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaa321-50af-436d-b771-2db16e18040f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:36.605009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaa321-50af-436d-b771-2db16e18040f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:36.605487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaa321-50af-436d-b771-2db16e18040f None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:36.606615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eaa321-50af-436d-b771-2db16e18040f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:36.607088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eaa321-50af-436d-b771-2db16e18040f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:36.608719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eaa321-50af-436d-b771-2db16e18040f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:36.622687 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 88/168] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:36.626432 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaa321-50af-436d-b771-2db16e18040f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:36.749028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaa321-50af-436d-b771-2db16e18040f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:36.759700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:36.790564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18848221-2352-4253-808e-84f5f5dc4f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:36.791969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18848221-2352-4253-808e-84f5f5dc4f31 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:36.792849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18848221-2352-4253-808e-84f5f5dc4f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:36.793301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18848221-2352-4253-808e-84f5f5dc4f31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:36.819717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:36.835457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3595c8b-cddd-4857-b1fc-6ca70d1979ac None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:36.882884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18848221-2352-4253-808e-84f5f5dc4f31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:36.885345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18848221-2352-4253-808e-84f5f5dc4f31 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:36.945664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e9b8f-9d91-4459-aeb5-4bdd8ddbcf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:36.946392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e9b8f-9d91-4459-aeb5-4bdd8ddbcf71 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:36.946771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e9b8f-9d91-4459-aeb5-4bdd8ddbcf71 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:36.947629 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e9b8f-9d91-4459-aeb5-4bdd8ddbcf71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.001524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 89/169] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:37.002030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18848221-2352-4253-808e-84f5f5dc4f31 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:37.031854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e9b8f-9d91-4459-aeb5-4bdd8ddbcf71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:37.033738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e9b8f-9d91-4459-aeb5-4bdd8ddbcf71 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:37.059879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:37.118696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18848221-2352-4253-808e-84f5f5dc4f31 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:37.123190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:37.153538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 81/170] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:37.154141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3a313-0555-41ee-9cc2-fb432d89d5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:37.154509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e9b8f-9d91-4459-aeb5-4bdd8ddbcf71 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:37.154799 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3a313-0555-41ee-9cc2-fb432d89d5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.155074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3a313-0555-41ee-9cc2-fb432d89d5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.155342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3a313-0555-41ee-9cc2-fb432d89d5e6 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.156536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3a313-0555-41ee-9cc2-fb432d89d5e6 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:37.158031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3a313-0555-41ee-9cc2-fb432d89d5e6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:37.159957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 90/171] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:37 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:37.161078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6887dc-1768-4c3d-a798-ed92cb066374 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.161356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6887dc-1768-4c3d-a798-ed92cb066374 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.161627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6887dc-1768-4c3d-a798-ed92cb066374 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.163287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 91/172] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:37.164027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6887dc-1768-4c3d-a798-ed92cb066374 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:37.272551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e9b8f-9d91-4459-aeb5-4bdd8ddbcf71 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:37.277252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6887dc-1768-4c3d-a798-ed92cb066374 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:37.301132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:37.312398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a431fb-6673-48aa-a79d-e74ce07bfa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:37.313028 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a431fb-6673-48aa-a79d-e74ce07bfa47 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.313305 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a431fb-6673-48aa-a79d-e74ce07bfa47 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.313570 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a431fb-6673-48aa-a79d-e74ce07bfa47 None admin] PATH_INFO: `/v3/users/02c8dc74a8e74fac884b7eaf15083780` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.314693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a431fb-6673-48aa-a79d-e74ce07bfa47 None admin] RBAC: Authorizing `identity:get_user(user_id=02c8dc74a8e74fac884b7eaf15083780)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:37.316155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a431fb-6673-48aa-a79d-e74ce07bfa47 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:37.317257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c849c1-9108-4a5e-bac1-7aebf96ca2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:37.317906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 82/173] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:37 2022] GET /identity/v3/users/02c8dc74a8e74fac884b7eaf15083780 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:37.318850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1e473-6510-42f6-9c03-7acb56c975cd None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.319126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1e473-6510-42f6-9c03-7acb56c975cd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.319395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1e473-6510-42f6-9c03-7acb56c975cd None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.325528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 83/174] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:37.325528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1e473-6510-42f6-9c03-7acb56c975cd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:37.325725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c849c1-9108-4a5e-bac1-7aebf96ca2cd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.325725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c849c1-9108-4a5e-bac1-7aebf96ca2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.325725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c849c1-9108-4a5e-bac1-7aebf96ca2cd None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.326140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c849c1-9108-4a5e-bac1-7aebf96ca2cd None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:37.327580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c849c1-9108-4a5e-bac1-7aebf96ca2cd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:37.332407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 92/175] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:37 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:37.332407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c849c1-9108-4a5e-bac1-7aebf96ca2cd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:37.377504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:37.433671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c849c1-9108-4a5e-bac1-7aebf96ca2cd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:37.436351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1e473-6510-42f6-9c03-7acb56c975cd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:37.466556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c404c-5280-4acf-bfcd-cec0e0c3c376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:37.467157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c404c-5280-4acf-bfcd-cec0e0c3c376 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.467435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c404c-5280-4acf-bfcd-cec0e0c3c376 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.467709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c404c-5280-4acf-bfcd-cec0e0c3c376 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.468447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c404c-5280-4acf-bfcd-cec0e0c3c376 None admin] RBAC: Adding query filter params (role.id=43452aaa2c9c4505b4bd3dd1b2412c65, scope.project.id=63a660324ea34aa886a24836744d158b, user.id=18593a37ac42418aab61de7fa0ece2c8) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:37.468746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74887e33-451b-42d0-abfd-1c6fc4c29c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:37.468868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c404c-5280-4acf-bfcd-cec0e0c3c376 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:37.469444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74887e33-451b-42d0-abfd-1c6fc4c29c8d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.469719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74887e33-451b-42d0-abfd-1c6fc4c29c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.469984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74887e33-451b-42d0-abfd-1c6fc4c29c8d None admin] PATH_INFO: `/v3/projects/a3396cb83fd94b628b0a5bc26b5c50c7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.470184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c404c-5280-4acf-bfcd-cec0e0c3c376 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:37.471081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74887e33-451b-42d0-abfd-1c6fc4c29c8d None admin] RBAC: Authorizing `identity:get_project(project_id=a3396cb83fd94b628b0a5bc26b5c50c7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:37.472538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74887e33-451b-42d0-abfd-1c6fc4c29c8d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:37.474142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 84/176] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:37 2022] GET /identity/v3/projects/a3396cb83fd94b628b0a5bc26b5c50c7 => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:37.475222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036560d-2298-4fe6-b612-7d4054242444 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.475498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036560d-2298-4fe6-b612-7d4054242444 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.486900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036560d-2298-4fe6-b612-7d4054242444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.487111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 93/177] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:37 2022] GET /identity/v3/role_assignments?user.id=18593a37ac42418aab61de7fa0ece2c8&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:37.487111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066c65a-a47c-4bd9-b98d-9ca4e8a93117 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.487111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066c65a-a47c-4bd9-b98d-9ca4e8a93117 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.487111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066c65a-a47c-4bd9-b98d-9ca4e8a93117 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.487111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 94/178] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:37.487111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f6fb6-44ba-434e-843c-828ccd73fb57 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.487530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f6fb6-44ba-434e-843c-828ccd73fb57 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.487530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f6fb6-44ba-434e-843c-828ccd73fb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.504455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4036560d-2298-4fe6-b612-7d4054242444 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:37.505700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4036560d-2298-4fe6-b612-7d4054242444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:37.510762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-902f6fb6-44ba-434e-843c-828ccd73fb57 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:37.512107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-902f6fb6-44ba-434e-843c-828ccd73fb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:37.567834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:37.583471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 85/179] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:37.585099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036560d-2298-4fe6-b612-7d4054242444 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:37.599160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 95/180] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:37.600150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f6fb6-44ba-434e-843c-828ccd73fb57 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:37.648775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:37.728179 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036560d-2298-4fe6-b612-7d4054242444 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:37.741652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f6fb6-44ba-434e-843c-828ccd73fb57 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:37.772003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31ce59-ddcc-49d0-9cac-a311f8027720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:37.772883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31ce59-ddcc-49d0-9cac-a311f8027720 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.773174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31ce59-ddcc-49d0-9cac-a311f8027720 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.773454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31ce59-ddcc-49d0-9cac-a311f8027720 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.784567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60bd2a-50bb-448b-be1e-613d40859474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:37.785172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60bd2a-50bb-448b-be1e-613d40859474 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.785442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60bd2a-50bb-448b-be1e-613d40859474 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.785707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60bd2a-50bb-448b-be1e-613d40859474 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.786423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60bd2a-50bb-448b-be1e-613d40859474 None admin] RBAC: Adding query filter params (role.id=9c17da6cc0e54b67af7cb7fb68638a7e, scope.project.id=63a660324ea34aa886a24836744d158b, user.id=18593a37ac42418aab61de7fa0ece2c8) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:37.786725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60bd2a-50bb-448b-be1e-613d40859474 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:37.788502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60bd2a-50bb-448b-be1e-613d40859474 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:37.794657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 96/181] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:37 2022] GET /identity/v3/role_assignments?user.id=18593a37ac42418aab61de7fa0ece2c8&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=9c17da6cc0e54b67af7cb7fb68638a7e => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:37.795831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff5b6a-cb4c-4604-90b6-6445e8e84801 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.796108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff5b6a-cb4c-4604-90b6-6445e8e84801 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.796392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff5b6a-cb4c-4604-90b6-6445e8e84801 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.798616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 97/182] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:37.799395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff5b6a-cb4c-4604-90b6-6445e8e84801 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:37.857539 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31ce59-ddcc-49d0-9cac-a311f8027720 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:37.859080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31ce59-ddcc-49d0-9cac-a311f8027720 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:37.863895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:37.913776 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:37.929840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff5b6a-cb4c-4604-90b6-6445e8e84801 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:37.978948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 86/183] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:37.979291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9824276-0831-4014-9464-888aa9877d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:37.979967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9824276-0831-4014-9464-888aa9877d34 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.980250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9824276-0831-4014-9464-888aa9877d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.980345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534268ba-5bd7-47c7-bead-617b093931fd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.980533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9824276-0831-4014-9464-888aa9877d34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.980622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534268ba-5bd7-47c7-bead-617b093931fd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.980816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534268ba-5bd7-47c7-bead-617b093931fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:37.981266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9824276-0831-4014-9464-888aa9877d34 None admin] RBAC: Adding query filter params (role.id=43452aaa2c9c4505b4bd3dd1b2412c65, scope.project.id=a3396cb83fd94b628b0a5bc26b5c50c7, user.id=18593a37ac42418aab61de7fa0ece2c8) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:37.981567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9824276-0831-4014-9464-888aa9877d34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:37.982992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9824276-0831-4014-9464-888aa9877d34 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:37.990700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 98/184] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:37 2022] GET /identity/v3/role_assignments?user.id=18593a37ac42418aab61de7fa0ece2c8&scope.project.id=a3396cb83fd94b628b0a5bc26b5c50c7&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:37.996426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14261496-2906-45a9-89d0-18df5675c408 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:37.996426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14261496-2906-45a9-89d0-18df5675c408 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:37.996426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14261496-2906-45a9-89d0-18df5675c408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:38.003158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-534268ba-5bd7-47c7-bead-617b093931fd None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:38.004428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-534268ba-5bd7-47c7-bead-617b093931fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:38.013520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-14261496-2906-45a9-89d0-18df5675c408 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:38.015520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-14261496-2906-45a9-89d0-18df5675c408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:38.086712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 87/185] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:38.087636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534268ba-5bd7-47c7-bead-617b093931fd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:38.114242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 99/186] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:38.115198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14261496-2906-45a9-89d0-18df5675c408 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:38.173558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:38.190730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:38.224998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534268ba-5bd7-47c7-bead-617b093931fd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:38.245739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14261496-2906-45a9-89d0-18df5675c408 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:38.264385 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297ccf4-5f1f-4533-be38-b3fd4ef1662a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:38.265020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297ccf4-5f1f-4533-be38-b3fd4ef1662a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:38.265296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297ccf4-5f1f-4533-be38-b3fd4ef1662a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:38.265566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297ccf4-5f1f-4533-be38-b3fd4ef1662a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:38.278874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297ccf4-5f1f-4533-be38-b3fd4ef1662a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a297ccf4-5f1f-4533-be38-b3fd4ef1662a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:38.284197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:38.285128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:38.285945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:38.286559 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 88/187] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:15:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:38.287347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297ccf4-5f1f-4533-be38-b3fd4ef1662a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:38.289266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede899f3-aeb1-4cb4-b8af-96b2e5d3597e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:38.290148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede899f3-aeb1-4cb4-b8af-96b2e5d3597e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:38.290719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede899f3-aeb1-4cb4-b8af-96b2e5d3597e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:38.291252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede899f3-aeb1-4cb4-b8af-96b2e5d3597e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:38.303064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede899f3-aeb1-4cb4-b8af-96b2e5d3597e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ede899f3-aeb1-4cb4-b8af-96b2e5d3597e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:38.310158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:38.311127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:38.311930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:38.314272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 100/188] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:15:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:38.315245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede899f3-aeb1-4cb4-b8af-96b2e5d3597e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:38.407826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297ccf4-5f1f-4533-be38-b3fd4ef1662a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:38.442768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:38.444893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede899f3-aeb1-4cb4-b8af-96b2e5d3597e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:38.448534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f41322-1adf-457f-8a39-5e3509015f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:38.448534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41322-1adf-457f-8a39-5e3509015f7f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:38.448763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:38.448837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41322-1adf-457f-8a39-5e3509015f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:38.449106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41322-1adf-457f-8a39-5e3509015f7f None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:38.450812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f41322-1adf-457f-8a39-5e3509015f7f None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:38.452616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f41322-1adf-457f-8a39-5e3509015f7f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:38.454929 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 89/189] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:38 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:38.456289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f41322-1adf-457f-8a39-5e3509015f7f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:38.497449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead1d93-f7ca-462e-818a-7eb92e4103e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:38.498067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead1d93-f7ca-462e-818a-7eb92e4103e5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:38.498347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead1d93-f7ca-462e-818a-7eb92e4103e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:38.498616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead1d93-f7ca-462e-818a-7eb92e4103e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:38.499284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead1d93-f7ca-462e-818a-7eb92e4103e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:38.499583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead1d93-f7ca-462e-818a-7eb92e4103e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:38.501090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead1d93-f7ca-462e-818a-7eb92e4103e5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:38.509851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 101/190] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:15:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:38.515036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead1d93-f7ca-462e-818a-7eb92e4103e5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:38.560612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f41322-1adf-457f-8a39-5e3509015f7f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:38.593716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34117389-c232-41c7-a01c-f337e9802b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:38.594345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34117389-c232-41c7-a01c-f337e9802b15 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:38.594625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34117389-c232-41c7-a01c-f337e9802b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:38.594896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34117389-c232-41c7-a01c-f337e9802b15 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:38.595557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34117389-c232-41c7-a01c-f337e9802b15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:38.595867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34117389-c232-41c7-a01c-f337e9802b15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:38.597273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34117389-c232-41c7-a01c-f337e9802b15 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:38.605955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 90/191] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:38.609781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34117389-c232-41c7-a01c-f337e9802b15 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:38.611476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead1d93-f7ca-462e-818a-7eb92e4103e5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:38.645192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549de98d-fccb-4245-8835-b5de68407653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:38.645821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549de98d-fccb-4245-8835-b5de68407653 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:38.646107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549de98d-fccb-4245-8835-b5de68407653 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:38.646372 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549de98d-fccb-4245-8835-b5de68407653 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:38.656551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549de98d-fccb-4245-8835-b5de68407653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-549de98d-fccb-4245-8835-b5de68407653 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:38.661728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:38.663034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:38.663838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 102/192] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:38.664536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549de98d-fccb-4245-8835-b5de68407653 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:38.670440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:38.688039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:38.709196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34117389-c232-41c7-a01c-f337e9802b15 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:38.742094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621ea5a-c301-4661-a8d1-5ea494b9fb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:38.742876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621ea5a-c301-4661-a8d1-5ea494b9fb94 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:38.743154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621ea5a-c301-4661-a8d1-5ea494b9fb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:38.743434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621ea5a-c301-4661-a8d1-5ea494b9fb94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:38.744216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621ea5a-c301-4661-a8d1-5ea494b9fb94 None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=63a660324ea34aa886a24836744d158b, user.id=02c8dc74a8e74fac884b7eaf15083780) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:38.744565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621ea5a-c301-4661-a8d1-5ea494b9fb94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:38.745994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621ea5a-c301-4661-a8d1-5ea494b9fb94 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:38.751690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 91/193] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:38 2022] GET /identity/v3/role_assignments?user.id=02c8dc74a8e74fac884b7eaf15083780&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:38.752568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621ea5a-c301-4661-a8d1-5ea494b9fb94 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:38.770054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549de98d-fccb-4245-8835-b5de68407653 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:38.809081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed917f-730a-4115-9930-997c8971a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:38.809743 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed917f-730a-4115-9930-997c8971a2ec None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:38.810022 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed917f-730a-4115-9930-997c8971a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:38.810293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed917f-730a-4115-9930-997c8971a2ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:38.811495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed917f-730a-4115-9930-997c8971a2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:38.813036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed917f-730a-4115-9930-997c8971a2ec None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:38.814874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 103/194] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:38.815528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed917f-730a-4115-9930-997c8971a2ec None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:38.873286 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621ea5a-c301-4661-a8d1-5ea494b9fb94 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:38.900732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:38.910962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609f75c-5711-4058-a380-5cc4dc072f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:38.911523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609f75c-5711-4058-a380-5cc4dc072f2b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:38.912045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609f75c-5711-4058-a380-5cc4dc072f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:38.912324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609f75c-5711-4058-a380-5cc4dc072f2b None admin] PATH_INFO: `/v3/users/3f1065c5d9334b2d9f2e5d75ce132835` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:38.919367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:38.928438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609f75c-5711-4058-a380-5cc4dc072f2b None admin] RBAC: Authorizing `identity:get_user(user_id=3f1065c5d9334b2d9f2e5d75ce132835)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:38.930021 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609f75c-5711-4058-a380-5cc4dc072f2b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:38.931756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 92/195] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:38 2022] GET /identity/v3/users/3f1065c5d9334b2d9f2e5d75ce132835 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:38.932533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed917f-730a-4115-9930-997c8971a2ec None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:38.937076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609f75c-5711-4058-a380-5cc4dc072f2b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:38.971030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a6a58-0550-4ea6-b2d1-022214ac877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:38.971962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a6a58-0550-4ea6-b2d1-022214ac877c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:38.972235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a6a58-0550-4ea6-b2d1-022214ac877c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:38.972542 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a6a58-0550-4ea6-b2d1-022214ac877c None admin] PATH_INFO: `/v3/users/3f1065c5d9334b2d9f2e5d75ce132835` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:38.974607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a6a58-0550-4ea6-b2d1-022214ac877c None admin] RBAC: Authorizing `identity:get_user(user_id=3f1065c5d9334b2d9f2e5d75ce132835)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:38.976054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a6a58-0550-4ea6-b2d1-022214ac877c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:38.981262 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 104/196] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:38 2022] GET /identity/v3/users/3f1065c5d9334b2d9f2e5d75ce132835 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:38.981262 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a6a58-0550-4ea6-b2d1-022214ac877c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:39.045792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609f75c-5711-4058-a380-5cc4dc072f2b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:39.080056 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0c1a0-15b3-4f64-8468-b3f54dfd9466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:39.080685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c1a0-15b3-4f64-8468-b3f54dfd9466 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:39.080960 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c1a0-15b3-4f64-8468-b3f54dfd9466 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:39.081228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c1a0-15b3-4f64-8468-b3f54dfd9466 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:39.081573 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a6a58-0550-4ea6-b2d1-022214ac877c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:39.081882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0c1a0-15b3-4f64-8468-b3f54dfd9466 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:39.082174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0c1a0-15b3-4f64-8468-b3f54dfd9466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:39.083922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0c1a0-15b3-4f64-8468-b3f54dfd9466 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:39.091983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 93/197] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:39.092804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0c1a0-15b3-4f64-8468-b3f54dfd9466 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:39.119927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c58ea-ec81-4652-9881-49bbd3eb6c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:39.120561 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c58ea-ec81-4652-9881-49bbd3eb6c02 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:39.120901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c58ea-ec81-4652-9881-49bbd3eb6c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:39.121195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c58ea-ec81-4652-9881-49bbd3eb6c02 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:39.121815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c58ea-ec81-4652-9881-49bbd3eb6c02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:39.123245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c58ea-ec81-4652-9881-49bbd3eb6c02 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:39.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:39.143408 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 105/198] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:15:38 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:39.144212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c58ea-ec81-4652-9881-49bbd3eb6c02 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:39.171211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:39.199363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0c1a0-15b3-4f64-8468-b3f54dfd9466 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:39.239877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48977356-02ed-4cf0-ae63-2894d8358fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:39.240527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48977356-02ed-4cf0-ae63-2894d8358fdf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:39.240805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48977356-02ed-4cf0-ae63-2894d8358fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:39.241072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48977356-02ed-4cf0-ae63-2894d8358fdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:39.272962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c58ea-ec81-4652-9881-49bbd3eb6c02 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:39.309368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2578a0f-8e19-4d75-a212-8def26782e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:39.309989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2578a0f-8e19-4d75-a212-8def26782e24 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:39.310260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2578a0f-8e19-4d75-a212-8def26782e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:39.310624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2578a0f-8e19-4d75-a212-8def26782e24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:39.326165 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48977356-02ed-4cf0-ae63-2894d8358fdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:39.327728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48977356-02ed-4cf0-ae63-2894d8358fdf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:39.377536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:39.391635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2578a0f-8e19-4d75-a212-8def26782e24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:39.393162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2578a0f-8e19-4d75-a212-8def26782e24 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:39.411775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:39.447967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 94/199] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:39.449104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48977356-02ed-4cf0-ae63-2894d8358fdf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:39.519990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 106/200] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:39.520842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2578a0f-8e19-4d75-a212-8def26782e24 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:39.568612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48977356-02ed-4cf0-ae63-2894d8358fdf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:39.604291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0286b2e-375e-4b22-9e92-8b2f79f8e52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:39.605096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0286b2e-375e-4b22-9e92-8b2f79f8e52e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:39.605487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0286b2e-375e-4b22-9e92-8b2f79f8e52e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:39.605850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0286b2e-375e-4b22-9e92-8b2f79f8e52e None admin] PATH_INFO: `/v3/users/ddb4a3ba691840c3a8ede64a3460318a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:39.607071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0286b2e-375e-4b22-9e92-8b2f79f8e52e None admin] RBAC: Authorizing `identity:get_user(user_id=ddb4a3ba691840c3a8ede64a3460318a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:39.608930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0286b2e-375e-4b22-9e92-8b2f79f8e52e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:39.610664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 95/201] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:39 2022] GET /identity/v3/users/ddb4a3ba691840c3a8ede64a3460318a => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:39.611545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0286b2e-375e-4b22-9e92-8b2f79f8e52e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:39.627878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:39.654969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2578a0f-8e19-4d75-a212-8def26782e24 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:39.675942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:39.704086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13958f-637b-42b8-bf4b-51f68f408818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:39.705362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13958f-637b-42b8-bf4b-51f68f408818 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:39.705790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13958f-637b-42b8-bf4b-51f68f408818 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:39.706205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13958f-637b-42b8-bf4b-51f68f408818 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:39.707539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13958f-637b-42b8-bf4b-51f68f408818 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:39.709984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13958f-637b-42b8-bf4b-51f68f408818 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:39.712795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 107/202] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:39 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:39.713798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13958f-637b-42b8-bf4b-51f68f408818 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:39.720986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0286b2e-375e-4b22-9e92-8b2f79f8e52e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:39.765533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eb30f-fea9-48a7-a96b-2af56bcb32d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:39.767311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eb30f-fea9-48a7-a96b-2af56bcb32d0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:39.767960 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eb30f-fea9-48a7-a96b-2af56bcb32d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:39.768564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eb30f-fea9-48a7-a96b-2af56bcb32d0 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:39.770237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eb30f-fea9-48a7-a96b-2af56bcb32d0 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:39.772565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eb30f-fea9-48a7-a96b-2af56bcb32d0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:39.780162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 96/203] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:39 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:39.780162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eb30f-fea9-48a7-a96b-2af56bcb32d0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:39.838146 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13958f-637b-42b8-bf4b-51f68f408818 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:39.876109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667201f1-e761-4212-80a3-ae145396701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:39.876901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667201f1-e761-4212-80a3-ae145396701c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:39.877268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667201f1-e761-4212-80a3-ae145396701c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:39.877628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667201f1-e761-4212-80a3-ae145396701c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:39.886879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:39.906336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eb30f-fea9-48a7-a96b-2af56bcb32d0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:39.933924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:39.941976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caaa85-2467-446b-bb10-7a1339833f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:39.942585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caaa85-2467-446b-bb10-7a1339833f50 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:39.942868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caaa85-2467-446b-bb10-7a1339833f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:39.943140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caaa85-2467-446b-bb10-7a1339833f50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:39.943868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51caaa85-2467-446b-bb10-7a1339833f50 None admin] RBAC: Adding query filter params (role.id=9c17da6cc0e54b67af7cb7fb68638a7e, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f, user.id=3f1065c5d9334b2d9f2e5d75ce132835) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:39.944176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51caaa85-2467-446b-bb10-7a1339833f50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:39.945642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51caaa85-2467-446b-bb10-7a1339833f50 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:39.951305 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 97/204] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:39 2022] GET /identity/v3/role_assignments?user.id=3f1065c5d9334b2d9f2e5d75ce132835&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=9c17da6cc0e54b67af7cb7fb68638a7e => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:39.952095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caaa85-2467-446b-bb10-7a1339833f50 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:39.957968 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667201f1-e761-4212-80a3-ae145396701c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:39.959884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667201f1-e761-4212-80a3-ae145396701c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:40.064676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caaa85-2467-446b-bb10-7a1339833f50 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:40.101800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad042607-0ea9-4a42-a65b-9b731c2d9153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:40.102421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042607-0ea9-4a42-a65b-9b731c2d9153 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:40.102700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042607-0ea9-4a42-a65b-9b731c2d9153 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:40.103527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042607-0ea9-4a42-a65b-9b731c2d9153 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:40.104272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad042607-0ea9-4a42-a65b-9b731c2d9153 None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f, user.id=3f1065c5d9334b2d9f2e5d75ce132835) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:40.104612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad042607-0ea9-4a42-a65b-9b731c2d9153 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:40.106086 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad042607-0ea9-4a42-a65b-9b731c2d9153 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:40.116619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 98/205] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:39 2022] GET /identity/v3/role_assignments?user.id=3f1065c5d9334b2d9f2e5d75ce132835&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:40.180509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 108/206] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:40.182970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad042607-0ea9-4a42-a65b-9b731c2d9153 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:40.206390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:40.312311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad042607-0ea9-4a42-a65b-9b731c2d9153 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:40.356656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ef6ad-f584-4a9e-9fc5-3727c3b0d5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:40.357295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef6ad-f584-4a9e-9fc5-3727c3b0d5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:40.357635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef6ad-f584-4a9e-9fc5-3727c3b0d5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:40.357906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef6ad-f584-4a9e-9fc5-3727c3b0d5a8 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:40.359096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ef6ad-f584-4a9e-9fc5-3727c3b0d5a8 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:40.360584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ef6ad-f584-4a9e-9fc5-3727c3b0d5a8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:40.362658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 99/207] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:40 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:40.368474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667201f1-e761-4212-80a3-ae145396701c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:40.468571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:40.494599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667201f1-e761-4212-80a3-ae145396701c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:40.537334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27bdc2-8f5d-471b-98fc-fc04012a9064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:40.537940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27bdc2-8f5d-471b-98fc-fc04012a9064 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:40.538217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27bdc2-8f5d-471b-98fc-fc04012a9064 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:40.538488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27bdc2-8f5d-471b-98fc-fc04012a9064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:40.539211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27bdc2-8f5d-471b-98fc-fc04012a9064 None admin] RBAC: Adding query filter params (role.id=fe9a4b7f28a04337bb3889d5fd2cf17d, scope.project.id=63a660324ea34aa886a24836744d158b, user.id=ddb4a3ba691840c3a8ede64a3460318a) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:40.539513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27bdc2-8f5d-471b-98fc-fc04012a9064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:40.541548 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27bdc2-8f5d-471b-98fc-fc04012a9064 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:40.553091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 109/208] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:40 2022] GET /identity/v3/role_assignments?user.id=ddb4a3ba691840c3a8ede64a3460318a&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=fe9a4b7f28a04337bb3889d5fd2cf17d => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:41.632905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053eb20a-6dfb-47a1-bd08-927ef8d65520 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:41.633951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053eb20a-6dfb-47a1-bd08-927ef8d65520 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:41.634990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053eb20a-6dfb-47a1-bd08-927ef8d65520 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:41.636899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 100/209] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:41.642876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d30838-0f16-4061-be6e-76e58060b242 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:41.643705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d30838-0f16-4061-be6e-76e58060b242 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:41.644175 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d30838-0f16-4061-be6e-76e58060b242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:41.662851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d6d30838-0f16-4061-be6e-76e58060b242 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:41.664316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d6d30838-0f16-4061-be6e-76e58060b242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:41.732314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41004441-1b28-425c-b0ee-3b173aee7017 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:41.732961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41004441-1b28-425c-b0ee-3b173aee7017 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:41.733458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41004441-1b28-425c-b0ee-3b173aee7017 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:41.736597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 101/210] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:41.740707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 110/211] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:41.741222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b240a7-46cc-48e0-9e2d-9956c984f74c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:41.741504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b240a7-46cc-48e0-9e2d-9956c984f74c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:41.741773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b240a7-46cc-48e0-9e2d-9956c984f74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:41.764286 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-94b240a7-46cc-48e0-9e2d-9956c984f74c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:41.765823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-94b240a7-46cc-48e0-9e2d-9956c984f74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:41.836631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 102/212] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:15:42.018956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d30838-0f16-4061-be6e-76e58060b242 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:42.098872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:42.106604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b240a7-46cc-48e0-9e2d-9956c984f74c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:42.116741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:42.123777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d30838-0f16-4061-be6e-76e58060b242 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:42.159052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93bfbb-d829-4f68-9ba8-b7dc5409da14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:42.159697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93bfbb-d829-4f68-9ba8-b7dc5409da14 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:42.159976 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93bfbb-d829-4f68-9ba8-b7dc5409da14 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:42.160249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93bfbb-d829-4f68-9ba8-b7dc5409da14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:42.171096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93bfbb-d829-4f68-9ba8-b7dc5409da14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-bb93bfbb-d829-4f68-9ba8-b7dc5409da14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:42.175936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:42.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:42.177669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 111/213] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:42.183502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93bfbb-d829-4f68-9ba8-b7dc5409da14 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:42.214981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b240a7-46cc-48e0-9e2d-9956c984f74c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:42.250301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2ba7-e6f8-4d9e-a175-2e366ee790e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:42.251071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2ba7-e6f8-4d9e-a175-2e366ee790e0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:42.251421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2ba7-e6f8-4d9e-a175-2e366ee790e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:42.251771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2ba7-e6f8-4d9e-a175-2e366ee790e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:42.253106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe2ba7-e6f8-4d9e-a175-2e366ee790e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:42.254712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe2ba7-e6f8-4d9e-a175-2e366ee790e0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:42.256976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 103/214] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:42.263336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2ba7-e6f8-4d9e-a175-2e366ee790e0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:42.313925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93bfbb-d829-4f68-9ba8-b7dc5409da14 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:42.353406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:42.354959 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8df8d-dab7-44e3-9890-fa79828a70f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:42.356197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8df8d-dab7-44e3-9890-fa79828a70f0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:42.356654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8df8d-dab7-44e3-9890-fa79828a70f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:42.357038 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8df8d-dab7-44e3-9890-fa79828a70f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:42.358090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8df8d-dab7-44e3-9890-fa79828a70f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:42.358492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8df8d-dab7-44e3-9890-fa79828a70f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:42.360039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8df8d-dab7-44e3-9890-fa79828a70f0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:42.362775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:42.371628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 112/215] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:42.377565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8df8d-dab7-44e3-9890-fa79828a70f0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:42.384806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2ba7-e6f8-4d9e-a175-2e366ee790e0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:42.424500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0edf4-c783-4037-a274-876110a70abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:42.425101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0edf4-c783-4037-a274-876110a70abe None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:42.425377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0edf4-c783-4037-a274-876110a70abe None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:42.425647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0edf4-c783-4037-a274-876110a70abe None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:42.426309 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0edf4-c783-4037-a274-876110a70abe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:42.427880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0edf4-c783-4037-a274-876110a70abe None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:42.449548 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 104/216] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:15:42 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:15:42.450937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d1ea-7e89-434d-97b5-acef3df7f50b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:42.451227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d1ea-7e89-434d-97b5-acef3df7f50b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:42.451499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d1ea-7e89-434d-97b5-acef3df7f50b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:42.452765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 105/217] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:42.458684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00cb5d-02a2-4126-be99-0dc9e375af41 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:42.459018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00cb5d-02a2-4126-be99-0dc9e375af41 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:42.459227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00cb5d-02a2-4126-be99-0dc9e375af41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:42.479884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-de00cb5d-02a2-4126-be99-0dc9e375af41 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:42.481200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-de00cb5d-02a2-4126-be99-0dc9e375af41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:42.512213 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8df8d-dab7-44e3-9890-fa79828a70f0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:42.565224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd100b41-a418-4102-aaa2-b1fc0cc85eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:42.566241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd100b41-a418-4102-aaa2-b1fc0cc85eae None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:42.566724 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd100b41-a418-4102-aaa2-b1fc0cc85eae None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:42.567204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd100b41-a418-4102-aaa2-b1fc0cc85eae None admin] PATH_INFO: `/v3/users/7a7a7487c7a84b28ab52e928d6468b4a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:42.568933 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 106/218] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:42.584038 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd100b41-a418-4102-aaa2-b1fc0cc85eae None admin] RBAC: Authorizing `identity:get_user(user_id=7a7a7487c7a84b28ab52e928d6468b4a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:42.585878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd100b41-a418-4102-aaa2-b1fc0cc85eae None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:42.588218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 113/219] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:42 2022] GET /identity/v3/users/7a7a7487c7a84b28ab52e928d6468b4a => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:42.596205 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00cb5d-02a2-4126-be99-0dc9e375af41 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:42.661147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:42.672064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1b35b-458f-4ce9-884a-76c9268d8d2c None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:42.672593 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1b35b-458f-4ce9-884a-76c9268d8d2c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:42.672684 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1b35b-458f-4ce9-884a-76c9268d8d2c None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:42.673980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 114/220] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:15:42.684088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8ab5-40a2-40ad-9326-fb99e01c6b4c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:42.684536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8ab5-40a2-40ad-9326-fb99e01c6b4c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:42.684949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8ab5-40a2-40ad-9326-fb99e01c6b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:42.707468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fb9f8ab5-40a2-40ad-9326-fb99e01c6b4c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:42.708816 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fb9f8ab5-40a2-40ad-9326-fb99e01c6b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:42.709370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00cb5d-02a2-4126-be99-0dc9e375af41 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:42.748977 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c585e9-def7-452a-8499-eaf8ed41c5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:42.749585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c585e9-def7-452a-8499-eaf8ed41c5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:42.749857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c585e9-def7-452a-8499-eaf8ed41c5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:42.750122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c585e9-def7-452a-8499-eaf8ed41c5a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:42.772833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:42.799288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 115/221] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:42.824722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c585e9-def7-452a-8499-eaf8ed41c5a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:42.826435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c585e9-def7-452a-8499-eaf8ed41c5a8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:42.871726 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f8ab5-40a2-40ad-9326-fb99e01c6b4c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:42.899117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:42.947070 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 107/222] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:42.952816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c585e9-def7-452a-8499-eaf8ed41c5a8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:43.000622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f8ab5-40a2-40ad-9326-fb99e01c6b4c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:43.038935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be7d03-27c6-4794-9c7f-ba69a41a2420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:43.039587 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be7d03-27c6-4794-9c7f-ba69a41a2420 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:43.039862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be7d03-27c6-4794-9c7f-ba69a41a2420 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:43.040133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be7d03-27c6-4794-9c7f-ba69a41a2420 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:43.053579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be7d03-27c6-4794-9c7f-ba69a41a2420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-65be7d03-27c6-4794-9c7f-ba69a41a2420 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:43.061030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:43.062166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:43.063067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:43.066688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 116/223] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:15:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:43.068463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be7d03-27c6-4794-9c7f-ba69a41a2420 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:43.084290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c585e9-def7-452a-8499-eaf8ed41c5a8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:43.121957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:43.127751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfe9c0-25d2-4d73-92b1-4737a0f2bba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:43.129099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe9c0-25d2-4d73-92b1-4737a0f2bba4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:43.129379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe9c0-25d2-4d73-92b1-4737a0f2bba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:43.129839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfe9c0-25d2-4d73-92b1-4737a0f2bba4 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:43.131854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfe9c0-25d2-4d73-92b1-4737a0f2bba4 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:43.135402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfe9c0-25d2-4d73-92b1-4737a0f2bba4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:43.139355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 108/224] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:42 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:43.140311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfe9c0-25d2-4d73-92b1-4737a0f2bba4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:43.173472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:43.183948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be7d03-27c6-4794-9c7f-ba69a41a2420 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:43.225944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966cd5c-0d0c-4bbf-882b-bf16a124ba1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:43.226553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966cd5c-0d0c-4bbf-882b-bf16a124ba1e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:43.226825 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966cd5c-0d0c-4bbf-882b-bf16a124ba1e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:43.227090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966cd5c-0d0c-4bbf-882b-bf16a124ba1e None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:43.227748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966cd5c-0d0c-4bbf-882b-bf16a124ba1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:43.229821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966cd5c-0d0c-4bbf-882b-bf16a124ba1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:43.230710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966cd5c-0d0c-4bbf-882b-bf16a124ba1e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:43.241602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 117/225] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:15:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:43.242718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966cd5c-0d0c-4bbf-882b-bf16a124ba1e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:43.268627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfe9c0-25d2-4d73-92b1-4737a0f2bba4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:43.304734 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bd65f-810b-4e51-991e-44346ffc9e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:43.305699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bd65f-810b-4e51-991e-44346ffc9e15 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:43.306283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bd65f-810b-4e51-991e-44346ffc9e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:43.306768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bd65f-810b-4e51-991e-44346ffc9e15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:43.319616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bd65f-810b-4e51-991e-44346ffc9e15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-140bd65f-810b-4e51-991e-44346ffc9e15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:43.327535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:43.328461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:43.329306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:43.336707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 109/226] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:15:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:43.337090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bd65f-810b-4e51-991e-44346ffc9e15 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:43.364083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966cd5c-0d0c-4bbf-882b-bf16a124ba1e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:43.374677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:43.405769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfccc40-dd6e-47cb-90f0-0965fb30779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:43.406363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfccc40-dd6e-47cb-90f0-0965fb30779c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:43.406643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfccc40-dd6e-47cb-90f0-0965fb30779c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:43.406914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfccc40-dd6e-47cb-90f0-0965fb30779c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:43.407648 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfccc40-dd6e-47cb-90f0-0965fb30779c None admin] RBAC: Adding query filter params (role.id=43452aaa2c9c4505b4bd3dd1b2412c65, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f, user.id=7a7a7487c7a84b28ab52e928d6468b4a) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:43.407938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfccc40-dd6e-47cb-90f0-0965fb30779c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:43.409417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfccc40-dd6e-47cb-90f0-0965fb30779c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:43.418492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 118/227] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:43 2022] GET /identity/v3/role_assignments?user.id=7a7a7487c7a84b28ab52e928d6468b4a&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:43.421184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfccc40-dd6e-47cb-90f0-0965fb30779c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:43.523643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:43.547694 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfccc40-dd6e-47cb-90f0-0965fb30779c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:43.550568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bd65f-810b-4e51-991e-44346ffc9e15 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:43.588635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d25a4-0d0c-4f23-b48b-530394261ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:43.589588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d25a4-0d0c-4f23-b48b-530394261ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:43.590058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d25a4-0d0c-4f23-b48b-530394261ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:43.590536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d25a4-0d0c-4f23-b48b-530394261ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:43.591402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d25a4-0d0c-4f23-b48b-530394261ae4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:43.591911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d25a4-0d0c-4f23-b48b-530394261ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:43.593659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d25a4-0d0c-4f23-b48b-530394261ae4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:43.602944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 119/228] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:43.609649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d25a4-0d0c-4f23-b48b-530394261ae4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:43.631198 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b17374-f294-4a89-a1e2-4264e6cdaf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:43.632443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b17374-f294-4a89-a1e2-4264e6cdaf13 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:43.632928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b17374-f294-4a89-a1e2-4264e6cdaf13 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:43.633537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b17374-f294-4a89-a1e2-4264e6cdaf13 None admin] PATH_INFO: `/v3/users/3f1065c5d9334b2d9f2e5d75ce132835` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:43.634913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b17374-f294-4a89-a1e2-4264e6cdaf13 None admin] RBAC: Authorizing `identity:get_user(user_id=3f1065c5d9334b2d9f2e5d75ce132835)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:43.637172 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b17374-f294-4a89-a1e2-4264e6cdaf13 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:43.639594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 110/229] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:43 2022] GET /identity/v3/users/3f1065c5d9334b2d9f2e5d75ce132835 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:43.646206 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b17374-f294-4a89-a1e2-4264e6cdaf13 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:43.657423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:43.767045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d25a4-0d0c-4f23-b48b-530394261ae4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:43.804539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc8a58-9de1-481f-b0c8-c8e26a931cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:43.805236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc8a58-9de1-481f-b0c8-c8e26a931cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:43.805517 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc8a58-9de1-481f-b0c8-c8e26a931cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:43.805794 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc8a58-9de1-481f-b0c8-c8e26a931cf0 None admin] PATH_INFO: `/v3/users/3f1065c5d9334b2d9f2e5d75ce132835` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:43.807595 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc8a58-9de1-481f-b0c8-c8e26a931cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=3f1065c5d9334b2d9f2e5d75ce132835)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:43.809075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc8a58-9de1-481f-b0c8-c8e26a931cf0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:43.811004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 120/230] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:43 2022] GET /identity/v3/users/3f1065c5d9334b2d9f2e5d75ce132835 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:43.816460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc8a58-9de1-481f-b0c8-c8e26a931cf0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:43.874292 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b17374-f294-4a89-a1e2-4264e6cdaf13 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:43.924021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc8a58-9de1-481f-b0c8-c8e26a931cf0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:43.936793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:43.947210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3786f0-dcb8-45eb-91ea-8b07e8c1222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:43.947876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3786f0-dcb8-45eb-91ea-8b07e8c1222b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:43.948127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3786f0-dcb8-45eb-91ea-8b07e8c1222b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:43.948419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3786f0-dcb8-45eb-91ea-8b07e8c1222b None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:43.949608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3786f0-dcb8-45eb-91ea-8b07e8c1222b None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:43.951172 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3786f0-dcb8-45eb-91ea-8b07e8c1222b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:43.953225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 111/231] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:43 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:43.965111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3786f0-dcb8-45eb-91ea-8b07e8c1222b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:43.970680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3257cd5-0e39-4994-8237-235208a985ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:43.971546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3257cd5-0e39-4994-8237-235208a985ca None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:43.971829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3257cd5-0e39-4994-8237-235208a985ca None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:43.972102 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3257cd5-0e39-4994-8237-235208a985ca None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:43.973338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3257cd5-0e39-4994-8237-235208a985ca None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:43.975825 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3257cd5-0e39-4994-8237-235208a985ca None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:43.978379 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 121/232] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:43 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:43.983640 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3257cd5-0e39-4994-8237-235208a985ca None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:44.020775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:44.099762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3257cd5-0e39-4994-8237-235208a985ca None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:44.138362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26227b2-ef4f-4ead-90c6-d5cc93ec9d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:44.138880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26227b2-ef4f-4ead-90c6-d5cc93ec9d36 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:44.139159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26227b2-ef4f-4ead-90c6-d5cc93ec9d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:44.139429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26227b2-ef4f-4ead-90c6-d5cc93ec9d36 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/users/3f1065c5d9334b2d9f2e5d75ce132835/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:44.141821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26227b2-ef4f-4ead-90c6-d5cc93ec9d36 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2c3fdc09a94be08d88fa3ebc44057f, user_id=3f1065c5d9334b2d9f2e5d75ce132835, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:44.143255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26227b2-ef4f-4ead-90c6-d5cc93ec9d36 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:44.162936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 122/233] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:15:43 2022] PUT /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/users/3f1065c5d9334b2d9f2e5d75ce132835/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:15:44.166993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3786f0-dcb8-45eb-91ea-8b07e8c1222b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:44.207610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680ffe4-b436-42d8-b1f9-916a719d8903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:44.208278 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ffe4-b436-42d8-b1f9-916a719d8903 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:44.208587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ffe4-b436-42d8-b1f9-916a719d8903 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:44.208861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680ffe4-b436-42d8-b1f9-916a719d8903 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/users/3f1065c5d9334b2d9f2e5d75ce132835/roles/9c17da6cc0e54b67af7cb7fb68638a7e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:44.211131 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680ffe4-b436-42d8-b1f9-916a719d8903 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2c3fdc09a94be08d88fa3ebc44057f, user_id=3f1065c5d9334b2d9f2e5d75ce132835, role_id=9c17da6cc0e54b67af7cb7fb68638a7e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:44.213255 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680ffe4-b436-42d8-b1f9-916a719d8903 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:44.240683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 112/234] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:15:43 2022] PUT /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/users/3f1065c5d9334b2d9f2e5d75ce132835/roles/9c17da6cc0e54b67af7cb7fb68638a7e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:15:45.095284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34106a1-cf31-4983-b2d9-8f18f8b43610 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:45.096127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34106a1-cf31-4983-b2d9-8f18f8b43610 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:45.096127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34106a1-cf31-4983-b2d9-8f18f8b43610 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:45.097882 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 123/235] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:45.110273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f0280-0245-4e02-abc8-8cb12d7ec6a6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:45.110825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f0280-0245-4e02-abc8-8cb12d7ec6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:45.110825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f0280-0245-4e02-abc8-8cb12d7ec6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:45.133225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e13f0280-0245-4e02-abc8-8cb12d7ec6a6 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:45.134461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e13f0280-0245-4e02-abc8-8cb12d7ec6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:45.225941 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 113/236] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:45.318329 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54826a-2888-4a80-9093-7f2d3598c31e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:45.318974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54826a-2888-4a80-9093-7f2d3598c31e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:45.319460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54826a-2888-4a80-9093-7f2d3598c31e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:45.321289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 124/237] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:45.326742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5664c5-5d50-4258-9004-a0d098f78516 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:45.327242 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5664c5-5d50-4258-9004-a0d098f78516 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:45.327699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5664c5-5d50-4258-9004-a0d098f78516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:45.346544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ad5664c5-5d50-4258-9004-a0d098f78516 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:45.347988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ad5664c5-5d50-4258-9004-a0d098f78516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:45.359549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:45.418398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 114/238] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:45.598257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54826a-2888-4a80-9093-7f2d3598c31e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:45.634725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:45.729602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54826a-2888-4a80-9093-7f2d3598c31e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:45.773168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581bf35-c6e6-431a-b99a-9cb409897f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:45.773802 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581bf35-c6e6-431a-b99a-9cb409897f2c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:45.774076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581bf35-c6e6-431a-b99a-9cb409897f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:45.774341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581bf35-c6e6-431a-b99a-9cb409897f2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:45.788323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5664c5-5d50-4258-9004-a0d098f78516 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:45.799756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581bf35-c6e6-431a-b99a-9cb409897f2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-4581bf35-c6e6-431a-b99a-9cb409897f2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:45.811711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:45.812731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:45.813530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:45.815258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 125/239] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:45.820261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581bf35-c6e6-431a-b99a-9cb409897f2c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:45.946208 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5664c5-5d50-4258-9004-a0d098f78516 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:45.960847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:45.973170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581bf35-c6e6-431a-b99a-9cb409897f2c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:45.988224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2f012-371b-405f-8e22-240613a87e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:45.989038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2f012-371b-405f-8e22-240613a87e0f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:45.989316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2f012-371b-405f-8e22-240613a87e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:45.989924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2f012-371b-405f-8e22-240613a87e0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:45.991299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2f012-371b-405f-8e22-240613a87e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:45.993184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2f012-371b-405f-8e22-240613a87e0f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:45.995325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 115/240] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:45.996436 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de47b-dbfd-4d08-9bc5-6543a2e049c0 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:45.997147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de47b-dbfd-4d08-9bc5-6543a2e049c0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:45.997484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de47b-dbfd-4d08-9bc5-6543a2e049c0 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:45.998995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 116/241] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:46.001114 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48de47b-dbfd-4d08-9bc5-6543a2e049c0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:46.012036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f96d6-3d5b-476c-a8c6-4eaca098066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:46.013310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f96d6-3d5b-476c-a8c6-4eaca098066e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:46.013979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f96d6-3d5b-476c-a8c6-4eaca098066e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:46.014344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f96d6-3d5b-476c-a8c6-4eaca098066e None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:46.015554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f96d6-3d5b-476c-a8c6-4eaca098066e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:46.015958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f96d6-3d5b-476c-a8c6-4eaca098066e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:46.018071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f96d6-3d5b-476c-a8c6-4eaca098066e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:46.020036 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:46.031075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 126/242] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:46.037838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d37fc-7771-4e92-b9dd-2c2d47ab539e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:46.038397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d37fc-7771-4e92-b9dd-2c2d47ab539e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:46.038676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d37fc-7771-4e92-b9dd-2c2d47ab539e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:46.061005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-355d37fc-7771-4e92-b9dd-2c2d47ab539e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:46.062788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-355d37fc-7771-4e92-b9dd-2c2d47ab539e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:46.122701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48de47b-dbfd-4d08-9bc5-6543a2e049c0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:46.142285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 127/243] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:46.143646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d37fc-7771-4e92-b9dd-2c2d47ab539e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:46.159068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d741d6-5209-4f68-b927-3debc35b96d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:46.159778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d741d6-5209-4f68-b927-3debc35b96d8 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:46.160058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d741d6-5209-4f68-b927-3debc35b96d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:46.160344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d741d6-5209-4f68-b927-3debc35b96d8 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:46.161325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d741d6-5209-4f68-b927-3debc35b96d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:46.162807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d741d6-5209-4f68-b927-3debc35b96d8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:46.191656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 117/244] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:15:46 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:46.239223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:46.260165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d37fc-7771-4e92-b9dd-2c2d47ab539e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:46.298331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f99b6-0447-4dc5-9891-608e1cf45810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:46.299261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f99b6-0447-4dc5-9891-608e1cf45810 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:46.299708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f99b6-0447-4dc5-9891-608e1cf45810 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:46.300174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f99b6-0447-4dc5-9891-608e1cf45810 None admin] PATH_INFO: `/v3/users/2e96750ad6674a378ed9d8fab296d9bd` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:46.315917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f99b6-0447-4dc5-9891-608e1cf45810 None admin] RBAC: Authorizing `identity:get_user(user_id=2e96750ad6674a378ed9d8fab296d9bd)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:46.317749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f99b6-0447-4dc5-9891-608e1cf45810 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:46.319983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 128/245] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:46 2022] GET /identity/v3/users/2e96750ad6674a378ed9d8fab296d9bd => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:46.329699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d741d6-5209-4f68-b927-3debc35b96d8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:46.444805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:46.483369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d741d6-5209-4f68-b927-3debc35b96d8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:46.498890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f99b6-0447-4dc5-9891-608e1cf45810 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:46.521868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2518610-8866-4cec-8773-10cb4eedcd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:46.522481 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2518610-8866-4cec-8773-10cb4eedcd89 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:46.523182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2518610-8866-4cec-8773-10cb4eedcd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:46.523458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2518610-8866-4cec-8773-10cb4eedcd89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:46.610153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2518610-8866-4cec-8773-10cb4eedcd89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:46.612349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2518610-8866-4cec-8773-10cb4eedcd89 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:46.633130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f99b6-0447-4dc5-9891-608e1cf45810 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:46.680223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b3265-357b-422d-8e48-3e46394b43ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:46.681422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b3265-357b-422d-8e48-3e46394b43ca None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:46.681704 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b3265-357b-422d-8e48-3e46394b43ca None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:46.681970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b3265-357b-422d-8e48-3e46394b43ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:46.697841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:46.701503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b3265-357b-422d-8e48-3e46394b43ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-723b3265-357b-422d-8e48-3e46394b43ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:46.706788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:46.707645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:46.708478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:46.709974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:46.711232 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 129/246] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:15:46.718054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b3265-357b-422d-8e48-3e46394b43ca None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:46.755951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 118/247] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:46.761793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2518610-8866-4cec-8773-10cb4eedcd89 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:46.868602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b3265-357b-422d-8e48-3e46394b43ca None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:46.887218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2518610-8866-4cec-8773-10cb4eedcd89 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:46.913511 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27a26a-2a5d-4729-b0d3-75b8402653bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:46.914505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27a26a-2a5d-4729-b0d3-75b8402653bd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:46.914975 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27a26a-2a5d-4729-b0d3-75b8402653bd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:46.916523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27a26a-2a5d-4729-b0d3-75b8402653bd None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:46.917656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27a26a-2a5d-4729-b0d3-75b8402653bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:46.918169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27a26a-2a5d-4729-b0d3-75b8402653bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:46.920536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27a26a-2a5d-4729-b0d3-75b8402653bd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:46.931291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66bae0-91b3-4f0d-93a6-f39edf0fa1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:46.932072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66bae0-91b3-4f0d-93a6-f39edf0fa1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:46.932613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66bae0-91b3-4f0d-93a6-f39edf0fa1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:46.932951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66bae0-91b3-4f0d-93a6-f39edf0fa1c5 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:46.933115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 130/248] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:46.938750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66bae0-91b3-4f0d-93a6-f39edf0fa1c5 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:46.938938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27a26a-2a5d-4729-b0d3-75b8402653bd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:46.940555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66bae0-91b3-4f0d-93a6-f39edf0fa1c5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:46.943058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 119/249] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:46 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:46.948779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66bae0-91b3-4f0d-93a6-f39edf0fa1c5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:46.997980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:47.024217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:47.070600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27a26a-2a5d-4729-b0d3-75b8402653bd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:47.073142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66bae0-91b3-4f0d-93a6-f39edf0fa1c5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:47.120643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b0ada-71c6-41c2-934b-0adda8da8ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:47.121257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b0ada-71c6-41c2-934b-0adda8da8ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:47.121533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b0ada-71c6-41c2-934b-0adda8da8ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:47.121803 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b0ada-71c6-41c2-934b-0adda8da8ef8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:47.122541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b0ada-71c6-41c2-934b-0adda8da8ef8 None admin] RBAC: Adding query filter params (role.id=fe9a4b7f28a04337bb3889d5fd2cf17d, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f, user.id=2e96750ad6674a378ed9d8fab296d9bd) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:47.122838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b0ada-71c6-41c2-934b-0adda8da8ef8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:47.124273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b0ada-71c6-41c2-934b-0adda8da8ef8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:47.125027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504a9d9-cf6d-4edb-b8e9-cea326de17c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:47.125662 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504a9d9-cf6d-4edb-b8e9-cea326de17c8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:47.125930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504a9d9-cf6d-4edb-b8e9-cea326de17c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:47.126192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504a9d9-cf6d-4edb-b8e9-cea326de17c8 None admin] PATH_INFO: `/v3/users/7a7a7487c7a84b28ab52e928d6468b4a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:47.129412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504a9d9-cf6d-4edb-b8e9-cea326de17c8 None admin] RBAC: Authorizing `identity:get_user(user_id=7a7a7487c7a84b28ab52e928d6468b4a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:47.130849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504a9d9-cf6d-4edb-b8e9-cea326de17c8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:47.132624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 131/250] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:46 2022] GET /identity/v3/users/7a7a7487c7a84b28ab52e928d6468b4a => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:47.139951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9f589-c496-4c2f-8335-4efc5733deb1 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:47.139951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9f589-c496-4c2f-8335-4efc5733deb1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:47.139951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9f589-c496-4c2f-8335-4efc5733deb1 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:47.139951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 132/251] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:47.141547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9f589-c496-4c2f-8335-4efc5733deb1 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:47.144202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 120/252] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:46 2022] GET /identity/v3/role_assignments?user.id=2e96750ad6674a378ed9d8fab296d9bd&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=fe9a4b7f28a04337bb3889d5fd2cf17d => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:47.145547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdd07c-e241-4fb2-99c1-4c1d21a178be None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:47.145836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdd07c-e241-4fb2-99c1-4c1d21a178be None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:47.146109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdd07c-e241-4fb2-99c1-4c1d21a178be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:47.165877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-60fdd07c-e241-4fb2-99c1-4c1d21a178be None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:47.167096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-60fdd07c-e241-4fb2-99c1-4c1d21a178be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:47.246708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 121/253] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:47.301115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9f589-c496-4c2f-8335-4efc5733deb1 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:47.333256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:47.340866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1c0e4-bc6e-4373-9479-c19a5dfdbb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:47.341872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1c0e4-bc6e-4373-9479-c19a5dfdbb06 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:47.342347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1c0e4-bc6e-4373-9479-c19a5dfdbb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:47.342851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1c0e4-bc6e-4373-9479-c19a5dfdbb06 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:47.344252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1c0e4-bc6e-4373-9479-c19a5dfdbb06 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:47.345926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1c0e4-bc6e-4373-9479-c19a5dfdbb06 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:47.348114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 133/254] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:47 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:47.353864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdd07c-e241-4fb2-99c1-4c1d21a178be None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:47.374721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb347b1c-01b6-4aa6-ada8-7c56e5ad9dbf None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:47.375028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb347b1c-01b6-4aa6-ada8-7c56e5ad9dbf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:47.375265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb347b1c-01b6-4aa6-ada8-7c56e5ad9dbf None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:47.376955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 134/255] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:47.382537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea32b28-51b2-46ca-b395-eed936b716c1 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:47.382832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea32b28-51b2-46ca-b395-eed936b716c1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:47.383268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea32b28-51b2-46ca-b395-eed936b716c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:47.404588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2ea32b28-51b2-46ca-b395-eed936b716c1 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:47.405807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2ea32b28-51b2-46ca-b395-eed936b716c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:47.433610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:47.511270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 135/256] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:47.515519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdd07c-e241-4fb2-99c1-4c1d21a178be None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:47.568157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679e5c0-af6e-4b28-82d2-d0d2333284ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:47.568795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679e5c0-af6e-4b28-82d2-d0d2333284ea None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:47.569075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679e5c0-af6e-4b28-82d2-d0d2333284ea None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:47.569345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679e5c0-af6e-4b28-82d2-d0d2333284ea None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/users/7a7a7487c7a84b28ab52e928d6468b4a/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:47.573621 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679e5c0-af6e-4b28-82d2-d0d2333284ea None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2c3fdc09a94be08d88fa3ebc44057f, user_id=7a7a7487c7a84b28ab52e928d6468b4a, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:47.575083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679e5c0-af6e-4b28-82d2-d0d2333284ea None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:47.595072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 122/257] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:15:47 2022] PUT /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/users/7a7a7487c7a84b28ab52e928d6468b4a/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:15:47.676563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea32b28-51b2-46ca-b395-eed936b716c1 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:47.809192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea32b28-51b2-46ca-b395-eed936b716c1 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:47.842866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:47.851715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679e5c0-af6e-4b28-82d2-d0d2333284ea None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:47.852108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e063b5-39d6-47c7-ac59-f5de9106dc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:47.853231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e063b5-39d6-47c7-ac59-f5de9106dc43 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:47.853612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e063b5-39d6-47c7-ac59-f5de9106dc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:47.854298 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e063b5-39d6-47c7-ac59-f5de9106dc43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:47.875036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e063b5-39d6-47c7-ac59-f5de9106dc43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-50e063b5-39d6-47c7-ac59-f5de9106dc43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:47.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:47.881887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:47.882691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 136/258] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:15:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:47.889332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e063b5-39d6-47c7-ac59-f5de9106dc43 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:48.009042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:48.033063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679e5c0-af6e-4b28-82d2-d0d2333284ea None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:48.087336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e063b5-39d6-47c7-ac59-f5de9106dc43 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:48.097664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226deb8-d29e-4dbb-85cb-8ec72f5a5e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:48.098566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226deb8-d29e-4dbb-85cb-8ec72f5a5e5a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:48.099018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226deb8-d29e-4dbb-85cb-8ec72f5a5e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:48.099462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226deb8-d29e-4dbb-85cb-8ec72f5a5e5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:48.116397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9226deb8-d29e-4dbb-85cb-8ec72f5a5e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9226deb8-d29e-4dbb-85cb-8ec72f5a5e5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:48.122873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:48.124191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:15:48.125126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:48.129558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 123/259] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:15:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:48.146528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226deb8-d29e-4dbb-85cb-8ec72f5a5e5a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:48.168466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf3e1d-cfae-4cd3-bc24-8a76c53ef795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:48.169346 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3e1d-cfae-4cd3-bc24-8a76c53ef795 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:48.169765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3e1d-cfae-4cd3-bc24-8a76c53ef795 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:48.170178 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3e1d-cfae-4cd3-bc24-8a76c53ef795 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:48.170977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf3e1d-cfae-4cd3-bc24-8a76c53ef795 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:48.171415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf3e1d-cfae-4cd3-bc24-8a76c53ef795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:48.172965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf3e1d-cfae-4cd3-bc24-8a76c53ef795 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:48.188998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 137/260] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:48.194894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf3e1d-cfae-4cd3-bc24-8a76c53ef795 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:48.286388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:48.325169 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226deb8-d29e-4dbb-85cb-8ec72f5a5e5a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:48.388106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110ec41-04c5-43a5-a75b-0f48f1f81723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:48.389228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110ec41-04c5-43a5-a75b-0f48f1f81723 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:48.390224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110ec41-04c5-43a5-a75b-0f48f1f81723 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:48.392781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110ec41-04c5-43a5-a75b-0f48f1f81723 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:48.393714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110ec41-04c5-43a5-a75b-0f48f1f81723 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:48.394191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110ec41-04c5-43a5-a75b-0f48f1f81723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:48.395774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110ec41-04c5-43a5-a75b-0f48f1f81723 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:48.395938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf3e1d-cfae-4cd3-bc24-8a76c53ef795 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:48.412767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 124/261] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:15:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:48.421801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110ec41-04c5-43a5-a75b-0f48f1f81723 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:48.458039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:48.458870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6b17e-7379-42c7-a79b-94624dcbca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:48.459832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b17e-7379-42c7-a79b-94624dcbca96 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:48.460111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b17e-7379-42c7-a79b-94624dcbca96 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:48.460590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b17e-7379-42c7-a79b-94624dcbca96 None admin] PATH_INFO: `/v3/users/3f1065c5d9334b2d9f2e5d75ce132835` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:48.462390 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6b17e-7379-42c7-a79b-94624dcbca96 None admin] RBAC: Authorizing `identity:get_user(user_id=3f1065c5d9334b2d9f2e5d75ce132835)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:48.465028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6b17e-7379-42c7-a79b-94624dcbca96 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:48.467012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 138/262] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:48 2022] GET /identity/v3/users/3f1065c5d9334b2d9f2e5d75ce132835 => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:48.473871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6b17e-7379-42c7-a79b-94624dcbca96 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:48.654575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110ec41-04c5-43a5-a75b-0f48f1f81723 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:48.654765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6b17e-7379-42c7-a79b-94624dcbca96 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:48.683980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:48.717972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c6ace-d419-4413-9499-f44f24203ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:48.719003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c6ace-d419-4413-9499-f44f24203ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:48.719287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c6ace-d419-4413-9499-f44f24203ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:48.719615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c6ace-d419-4413-9499-f44f24203ee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:48.739526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540e8bf-acef-4355-845b-f47e21d1eaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:48.740170 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540e8bf-acef-4355-845b-f47e21d1eaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:48.740548 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540e8bf-acef-4355-845b-f47e21d1eaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:48.740815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540e8bf-acef-4355-845b-f47e21d1eaf2 None admin] PATH_INFO: `/v3/users/3f1065c5d9334b2d9f2e5d75ce132835` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:48.745093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540e8bf-acef-4355-845b-f47e21d1eaf2 None admin] RBAC: Authorizing `identity:get_user(user_id=3f1065c5d9334b2d9f2e5d75ce132835)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:48.746591 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540e8bf-acef-4355-845b-f47e21d1eaf2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:48.748524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 125/263] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:48 2022] GET /identity/v3/users/3f1065c5d9334b2d9f2e5d75ce132835 => generated 283 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:48.756974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540e8bf-acef-4355-845b-f47e21d1eaf2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:48.854001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c6ace-d419-4413-9499-f44f24203ee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:48.855477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c6ace-d419-4413-9499-f44f24203ee7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:48.933772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:48.955877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540e8bf-acef-4355-845b-f47e21d1eaf2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:48.991246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9ec2d-349d-4c70-a058-c1e3d4875635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:48.991856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9ec2d-349d-4c70-a058-c1e3d4875635 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:48.992140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9ec2d-349d-4c70-a058-c1e3d4875635 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:48.992439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9ec2d-349d-4c70-a058-c1e3d4875635 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:49.014387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 139/264] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:49.019648 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c6ace-d419-4413-9499-f44f24203ee7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:49.050141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:49.119576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9ec2d-349d-4c70-a058-c1e3d4875635 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:49.125303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9ec2d-349d-4c70-a058-c1e3d4875635 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:49.198012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c6ace-d419-4413-9499-f44f24203ee7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:49.264187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef75153-7cd8-4c50-b307-171f69a42089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:49.264854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef75153-7cd8-4c50-b307-171f69a42089 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:49.265128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef75153-7cd8-4c50-b307-171f69a42089 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:49.265395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef75153-7cd8-4c50-b307-171f69a42089 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:49.266538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef75153-7cd8-4c50-b307-171f69a42089 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:49.267974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef75153-7cd8-4c50-b307-171f69a42089 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:49.269603 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 140/265] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:49 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:49.274369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:49.275565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef75153-7cd8-4c50-b307-171f69a42089 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:49.311455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 126/266] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 30 19:15:49.317453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9ec2d-349d-4c70-a058-c1e3d4875635 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:49.433044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:49.459233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef75153-7cd8-4c50-b307-171f69a42089 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:49.518444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ca468-dead-4a4d-91c4-5b580a3ef3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:49.519047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca468-dead-4a4d-91c4-5b580a3ef3be None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:49.519328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca468-dead-4a4d-91c4-5b580a3ef3be None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:49.519600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ca468-dead-4a4d-91c4-5b580a3ef3be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:49.520328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ca468-dead-4a4d-91c4-5b580a3ef3be None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f, user.id=3f1065c5d9334b2d9f2e5d75ce132835) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:49.520659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ca468-dead-4a4d-91c4-5b580a3ef3be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:49.522105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ca468-dead-4a4d-91c4-5b580a3ef3be None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:49.534941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 141/267] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:49 2022] GET /identity/v3/role_assignments?user.id=3f1065c5d9334b2d9f2e5d75ce132835&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:49.539161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9ec2d-349d-4c70-a058-c1e3d4875635 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:49.598535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914678f-f34d-46cb-9e67-872852cdc067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:49.599241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914678f-f34d-46cb-9e67-872852cdc067 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:49.599519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914678f-f34d-46cb-9e67-872852cdc067 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:49.599840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914678f-f34d-46cb-9e67-872852cdc067 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:49.603276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914678f-f34d-46cb-9e67-872852cdc067 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:49.604754 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914678f-f34d-46cb-9e67-872852cdc067 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:49.606775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 127/268] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:49 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:49.612755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ca468-dead-4a4d-91c4-5b580a3ef3be None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:49.723897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ca468-dead-4a4d-91c4-5b580a3ef3be None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:49.760500 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b16c2e-884d-4cf4-a7bc-0511dc7983b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:49.761131 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b16c2e-884d-4cf4-a7bc-0511dc7983b9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:49.761413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b16c2e-884d-4cf4-a7bc-0511dc7983b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:49.761688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b16c2e-884d-4cf4-a7bc-0511dc7983b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:49.762425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b16c2e-884d-4cf4-a7bc-0511dc7983b9 None admin] RBAC: Adding query filter params (role.id=9c17da6cc0e54b67af7cb7fb68638a7e, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f, user.id=3f1065c5d9334b2d9f2e5d75ce132835) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:49.762733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b16c2e-884d-4cf4-a7bc-0511dc7983b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:49.764201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b16c2e-884d-4cf4-a7bc-0511dc7983b9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:49.773364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 142/269] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:49 2022] GET /identity/v3/role_assignments?user.id=3f1065c5d9334b2d9f2e5d75ce132835&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=9c17da6cc0e54b67af7cb7fb68638a7e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:50.125942 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d8508-66f7-4106-9e4c-134abda45408 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:50.126756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d8508-66f7-4106-9e4c-134abda45408 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:50.127180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d8508-66f7-4106-9e4c-134abda45408 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:50.128971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 128/270] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:50.135779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee0796-56a0-4de5-909f-39a381238bb0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:50.136154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee0796-56a0-4de5-909f-39a381238bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:50.136463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee0796-56a0-4de5-909f-39a381238bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:50.161733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b5ee0796-56a0-4de5-909f-39a381238bb0 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:50.163500 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b5ee0796-56a0-4de5-909f-39a381238bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:50.176087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:50.180260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996e925-bc99-40cb-8839-aef2360c8b83 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:50.180945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996e925-bc99-40cb-8839-aef2360c8b83 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:50.181414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996e925-bc99-40cb-8839-aef2360c8b83 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:50.183034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 129/271] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:50.190856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12352d5-1a89-4235-8c96-c2d4689be5ea None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:50.191459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12352d5-1a89-4235-8c96-c2d4689be5ea None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:50.191943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12352d5-1a89-4235-8c96-c2d4689be5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:50.215665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c12352d5-1a89-4235-8c96-c2d4689be5ea None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:50.218246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c12352d5-1a89-4235-8c96-c2d4689be5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:50.254718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 143/272] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:50.272810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:50.297050 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 130/273] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:50.311561 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eef389-adf9-4333-adaf-5565074be18d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:50.312044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eef389-adf9-4333-adaf-5565074be18d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:50.312490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eef389-adf9-4333-adaf-5565074be18d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:50.313991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 144/274] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:50.319501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd622-805f-4d3d-b75c-db9b8ad12f5f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:50.319786 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd622-805f-4d3d-b75c-db9b8ad12f5f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:50.320054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd622-805f-4d3d-b75c-db9b8ad12f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:50.341326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b36cd622-805f-4d3d-b75c-db9b8ad12f5f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:50.342499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b36cd622-805f-4d3d-b75c-db9b8ad12f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:50.424465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 131/275] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:50.551530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eef389-adf9-4333-adaf-5565074be18d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:50.586832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cd622-805f-4d3d-b75c-db9b8ad12f5f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:50.663733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eef389-adf9-4333-adaf-5565074be18d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:50.701144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cd622-805f-4d3d-b75c-db9b8ad12f5f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:50.703313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee9986-75e3-42e0-b6bc-99e976eeb4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:50.704985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee9986-75e3-42e0-b6bc-99e976eeb4ae None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:50.705455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee9986-75e3-42e0-b6bc-99e976eeb4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:50.705931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee9986-75e3-42e0-b6bc-99e976eeb4ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:50.707233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee9986-75e3-42e0-b6bc-99e976eeb4ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:50.708884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee9986-75e3-42e0-b6bc-99e976eeb4ae None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:50.711734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 145/276] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:50.716642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:50.718453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee9986-75e3-42e0-b6bc-99e976eeb4ae None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:50.734618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:50.745854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef088d34-c700-443a-9cd2-dd927a860393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:50.746783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef088d34-c700-443a-9cd2-dd927a860393 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:50.747200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef088d34-c700-443a-9cd2-dd927a860393 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:50.747625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef088d34-c700-443a-9cd2-dd927a860393 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:50.759302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef088d34-c700-443a-9cd2-dd927a860393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ef088d34-c700-443a-9cd2-dd927a860393 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:50.764499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:50.765579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 132/277] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:15:50.770051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef088d34-c700-443a-9cd2-dd927a860393 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:50.834147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee9986-75e3-42e0-b6bc-99e976eeb4ae None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:50.870711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b2f32-39fe-4b5c-96fb-aefb58d5c3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:50.871559 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b2f32-39fe-4b5c-96fb-aefb58d5c3d1 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:50.871850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b2f32-39fe-4b5c-96fb-aefb58d5c3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:50.872101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b2f32-39fe-4b5c-96fb-aefb58d5c3d1 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:50.872767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b2f32-39fe-4b5c-96fb-aefb58d5c3d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:50.874196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b2f32-39fe-4b5c-96fb-aefb58d5c3d1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:50.881238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef088d34-c700-443a-9cd2-dd927a860393 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:50.895757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 146/278] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:15:50 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:50.896592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b2f32-39fe-4b5c-96fb-aefb58d5c3d1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:50.919441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae27794-ba6a-47ff-812a-ffa947687edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:50.920013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae27794-ba6a-47ff-812a-ffa947687edd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:50.920300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae27794-ba6a-47ff-812a-ffa947687edd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:50.920596 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae27794-ba6a-47ff-812a-ffa947687edd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:50.932956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae27794-ba6a-47ff-812a-ffa947687edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bae27794-ba6a-47ff-812a-ffa947687edd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:50.938737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:50.939762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:50.940620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 133/279] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:50.941293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0151c0-20e8-4b30-b8d1-0cfb78cb9964 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:50.941427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0151c0-20e8-4b30-b8d1-0cfb78cb9964 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:50.941701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0151c0-20e8-4b30-b8d1-0cfb78cb9964 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:50.944463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 134/280] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:50.949867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0151c0-20e8-4b30-b8d1-0cfb78cb9964 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:50.983631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:50.986894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:51.028309 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b2f32-39fe-4b5c-96fb-aefb58d5c3d1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:51.069136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0151c0-20e8-4b30-b8d1-0cfb78cb9964 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:51.070794 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe265d3-c6c6-4674-bfce-380cab05e2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:51.071441 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe265d3-c6c6-4674-bfce-380cab05e2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:51.071718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe265d3-c6c6-4674-bfce-380cab05e2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:51.073069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe265d3-c6c6-4674-bfce-380cab05e2a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:51.073069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe265d3-c6c6-4674-bfce-380cab05e2a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:51.073069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe265d3-c6c6-4674-bfce-380cab05e2a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:51.074879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe265d3-c6c6-4674-bfce-380cab05e2a0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:51.084436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 147/281] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:51.085480 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727bd3a-7f33-471b-8263-50cdfe67fad8 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:51.085765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727bd3a-7f33-471b-8263-50cdfe67fad8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:51.086036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727bd3a-7f33-471b-8263-50cdfe67fad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:51.109719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e727bd3a-7f33-471b-8263-50cdfe67fad8 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:51.110887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e727bd3a-7f33-471b-8263-50cdfe67fad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:51.113869 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd12e4-0198-4922-b497-2afb7f35a623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:51.114489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd12e4-0198-4922-b497-2afb7f35a623 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:51.114771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd12e4-0198-4922-b497-2afb7f35a623 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:51.115110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd12e4-0198-4922-b497-2afb7f35a623 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:51.115786 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd12e4-0198-4922-b497-2afb7f35a623 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:51.119719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd12e4-0198-4922-b497-2afb7f35a623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:51.119719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd12e4-0198-4922-b497-2afb7f35a623 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:51.127818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 135/282] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:51.128671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd12e4-0198-4922-b497-2afb7f35a623 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:51.201621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 148/283] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:51.202318 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727bd3a-7f33-471b-8263-50cdfe67fad8 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:51.248221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:51.258350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd12e4-0198-4922-b497-2afb7f35a623 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:51.277663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:51.307913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2a8af-ed9c-4a80-8e91-afe3a49fa53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:51.308583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2a8af-ed9c-4a80-8e91-afe3a49fa53c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:51.308857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2a8af-ed9c-4a80-8e91-afe3a49fa53c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:51.309124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2a8af-ed9c-4a80-8e91-afe3a49fa53c None admin] PATH_INFO: `/v3/users/d316133186494183b49d933eb9c36f70` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:51.329201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2a8af-ed9c-4a80-8e91-afe3a49fa53c None admin] RBAC: Authorizing `identity:get_user(user_id=d316133186494183b49d933eb9c36f70)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:51.331374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2a8af-ed9c-4a80-8e91-afe3a49fa53c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:51.332321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727bd3a-7f33-471b-8263-50cdfe67fad8 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:51.337965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 136/284] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:51 2022] GET /identity/v3/users/d316133186494183b49d933eb9c36f70 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:51.338832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2a8af-ed9c-4a80-8e91-afe3a49fa53c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:51.376121 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45a793-419f-4e95-a2f9-a8a1ab4c1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:51.377331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45a793-419f-4e95-a2f9-a8a1ab4c1c88 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:51.377808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45a793-419f-4e95-a2f9-a8a1ab4c1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:51.379105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45a793-419f-4e95-a2f9-a8a1ab4c1c88 None admin] PATH_INFO: `/v3/users/2e96750ad6674a378ed9d8fab296d9bd` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:51.380487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45a793-419f-4e95-a2f9-a8a1ab4c1c88 None admin] RBAC: Authorizing `identity:get_user(user_id=2e96750ad6674a378ed9d8fab296d9bd)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:51.382106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45a793-419f-4e95-a2f9-a8a1ab4c1c88 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:51.384996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 149/285] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:51 2022] GET /identity/v3/users/2e96750ad6674a378ed9d8fab296d9bd => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:51.393707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45a793-419f-4e95-a2f9-a8a1ab4c1c88 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:51.453330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2a8af-ed9c-4a80-8e91-afe3a49fa53c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:51.493161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135ec29-1c54-4e50-a73e-bb10861c9569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:51.494302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135ec29-1c54-4e50-a73e-bb10861c9569 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:51.494778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135ec29-1c54-4e50-a73e-bb10861c9569 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:51.495256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135ec29-1c54-4e50-a73e-bb10861c9569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:51.496294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135ec29-1c54-4e50-a73e-bb10861c9569 None admin] RBAC: Adding query filter params (role.id=43452aaa2c9c4505b4bd3dd1b2412c65, user.id=d316133186494183b49d933eb9c36f70, scope.system=all) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:51.496934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135ec29-1c54-4e50-a73e-bb10861c9569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:51.498572 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135ec29-1c54-4e50-a73e-bb10861c9569 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:51.504234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 137/286] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 30 19:15:51 2022] GET /identity/v3/role_assignments?user.id=d316133186494183b49d933eb9c36f70&scope.system=all&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:51.505459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135ec29-1c54-4e50-a73e-bb10861c9569 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:51.523898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:51.557627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45a793-419f-4e95-a2f9-a8a1ab4c1c88 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:51.589106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:51.608416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a4d2e-f890-4255-a583-a60d92d47639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:51.608416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a4d2e-f890-4255-a583-a60d92d47639 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:51.608416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a4d2e-f890-4255-a583-a60d92d47639 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:51.608416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a4d2e-f890-4255-a583-a60d92d47639 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:51.610133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a4d2e-f890-4255-a583-a60d92d47639 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:51.611858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a4d2e-f890-4255-a583-a60d92d47639 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:51.614059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 150/287] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:51 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:51.620949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a4d2e-f890-4255-a583-a60d92d47639 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:51.666813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135ec29-1c54-4e50-a73e-bb10861c9569 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:51.746093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880c7fa-d255-4677-9eee-e0b7b929a2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:51.746661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880c7fa-d255-4677-9eee-e0b7b929a2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:51.746936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880c7fa-d255-4677-9eee-e0b7b929a2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:51.747201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880c7fa-d255-4677-9eee-e0b7b929a2b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:51.773379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0880c7fa-d255-4677-9eee-e0b7b929a2b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0880c7fa-d255-4677-9eee-e0b7b929a2b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:51.779226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:51.780108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:51.780987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:51.785272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 138/288] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:51.788560 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a4d2e-f890-4255-a583-a60d92d47639 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:51.789269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880c7fa-d255-4677-9eee-e0b7b929a2b8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:51.823987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37531d05-a0a8-40f1-bc4d-6656e78752e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:51.824685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37531d05-a0a8-40f1-bc4d-6656e78752e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:51.824960 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37531d05-a0a8-40f1-bc4d-6656e78752e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:51.825231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37531d05-a0a8-40f1-bc4d-6656e78752e1 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/users/2e96750ad6674a378ed9d8fab296d9bd/roles/fe9a4b7f28a04337bb3889d5fd2cf17d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:51.827140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37531d05-a0a8-40f1-bc4d-6656e78752e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2c3fdc09a94be08d88fa3ebc44057f, user_id=2e96750ad6674a378ed9d8fab296d9bd, role_id=fe9a4b7f28a04337bb3889d5fd2cf17d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:51.828618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37531d05-a0a8-40f1-bc4d-6656e78752e1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:51.843799 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 151/289] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:15:51 2022] PUT /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/users/2e96750ad6674a378ed9d8fab296d9bd/roles/fe9a4b7f28a04337bb3889d5fd2cf17d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:15:51.944894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:51.979704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880c7fa-d255-4677-9eee-e0b7b929a2b8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:52.018200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a1ca7-b72b-4ebb-9f79-b53e8bb6de29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:52.018733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a1ca7-b72b-4ebb-9f79-b53e8bb6de29 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:52.019011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a1ca7-b72b-4ebb-9f79-b53e8bb6de29 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:52.019281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a1ca7-b72b-4ebb-9f79-b53e8bb6de29 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:52.019954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a1ca7-b72b-4ebb-9f79-b53e8bb6de29 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:52.020256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a1ca7-b72b-4ebb-9f79-b53e8bb6de29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:52.021704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a1ca7-b72b-4ebb-9f79-b53e8bb6de29 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:52.031524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 139/290] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:52.037258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37531d05-a0a8-40f1-bc4d-6656e78752e1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:52.099010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:52.148493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37531d05-a0a8-40f1-bc4d-6656e78752e1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:52.185510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c569-ba30-4934-bc18-e3219114ebd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:52.186383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c569-ba30-4934-bc18-e3219114ebd9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:52.186797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c569-ba30-4934-bc18-e3219114ebd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:52.187204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c569-ba30-4934-bc18-e3219114ebd9 None admin] PATH_INFO: `/v3/users/7a7a7487c7a84b28ab52e928d6468b4a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:52.188509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4c569-ba30-4934-bc18-e3219114ebd9 None admin] RBAC: Authorizing `identity:get_user(user_id=7a7a7487c7a84b28ab52e928d6468b4a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:52.191516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4c569-ba30-4934-bc18-e3219114ebd9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:52.193789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 152/291] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:52 2022] GET /identity/v3/users/7a7a7487c7a84b28ab52e928d6468b4a => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:52.199821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a1ca7-b72b-4ebb-9f79-b53e8bb6de29 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:52.336699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a1ca7-b72b-4ebb-9f79-b53e8bb6de29 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:52.371053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008afbc-25dd-42e3-a0be-db3831855ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:52.371659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008afbc-25dd-42e3-a0be-db3831855ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:52.371937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008afbc-25dd-42e3-a0be-db3831855ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:52.372211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008afbc-25dd-42e3-a0be-db3831855ca2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:52.375723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:52.445871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008afbc-25dd-42e3-a0be-db3831855ca2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:52.447869 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008afbc-25dd-42e3-a0be-db3831855ca2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:52.569046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 140/292] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:52.574483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c569-ba30-4934-bc18-e3219114ebd9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:52.728538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c569-ba30-4934-bc18-e3219114ebd9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:52.765374 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:52.777758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39d8b3-6a65-4bf8-9073-b6b23f629ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:52.778694 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39d8b3-6a65-4bf8-9073-b6b23f629ded None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:52.779169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39d8b3-6a65-4bf8-9073-b6b23f629ded None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:52.779636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39d8b3-6a65-4bf8-9073-b6b23f629ded None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:52.780992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39d8b3-6a65-4bf8-9073-b6b23f629ded None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:52.782614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39d8b3-6a65-4bf8-9073-b6b23f629ded None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:52.784809 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 153/293] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:52 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:52.790834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008afbc-25dd-42e3-a0be-db3831855ca2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:52.848185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:52.942308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008afbc-25dd-42e3-a0be-db3831855ca2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:52.991874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b066b76-b65e-47da-82a4-3df231d44c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:52.992416 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066b76-b65e-47da-82a4-3df231d44c60 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:52.992700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066b76-b65e-47da-82a4-3df231d44c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:52.992971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066b76-b65e-47da-82a4-3df231d44c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:52.994260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b066b76-b65e-47da-82a4-3df231d44c60 None admin] RBAC: Adding query filter params (role.id=43452aaa2c9c4505b4bd3dd1b2412c65, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f, user.id=7a7a7487c7a84b28ab52e928d6468b4a) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:52.994565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b066b76-b65e-47da-82a4-3df231d44c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:52.995987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b066b76-b65e-47da-82a4-3df231d44c60 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:53.004771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 141/294] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:52 2022] GET /identity/v3/role_assignments?user.id=7a7a7487c7a84b28ab52e928d6468b4a&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:53.649890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6329c5-6cac-4193-bb06-c8e5b56270c1 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:53.650914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6329c5-6cac-4193-bb06-c8e5b56270c1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:53.651409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6329c5-6cac-4193-bb06-c8e5b56270c1 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:53.653130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 154/295] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:53.658042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd956c-578f-4f02-948c-e4df447eb37f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:53.658452 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd956c-578f-4f02-948c-e4df447eb37f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:53.658639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd956c-578f-4f02-948c-e4df447eb37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:53.682291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d3dd956c-578f-4f02-948c-e4df447eb37f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:53.683543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d3dd956c-578f-4f02-948c-e4df447eb37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:53.777175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 142/296] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:53.810511 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fec66-c8c1-4935-912a-5a5353126a54 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:53.811210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fec66-c8c1-4935-912a-5a5353126a54 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:53.811680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fec66-c8c1-4935-912a-5a5353126a54 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:53.813490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 155/297] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:53.818709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff7642-d19b-4e70-a3d0-c1efe683979f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:53.818997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff7642-d19b-4e70-a3d0-c1efe683979f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:53.819509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff7642-d19b-4e70-a3d0-c1efe683979f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:53.838387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d1ff7642-d19b-4e70-a3d0-c1efe683979f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:53.839862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d1ff7642-d19b-4e70-a3d0-c1efe683979f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:53.864706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:53.908496 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 143/298] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:54.052263 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fec66-c8c1-4935-912a-5a5353126a54 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:54.162567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fec66-c8c1-4935-912a-5a5353126a54 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:54.177721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff7642-d19b-4e70-a3d0-c1efe683979f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:54.198112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a34c5-1de6-4ded-98eb-1f98a5278869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:54.199055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a34c5-1de6-4ded-98eb-1f98a5278869 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:54.199547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a34c5-1de6-4ded-98eb-1f98a5278869 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:54.200007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a34c5-1de6-4ded-98eb-1f98a5278869 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:54.210728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a34c5-1de6-4ded-98eb-1f98a5278869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-414a34c5-1de6-4ded-98eb-1f98a5278869 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:54.216326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:54.217228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:54.218059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:54.220524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 156/299] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:54.222427 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c89478-ea79-4ad5-acdd-034b90b701bc None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:54.222914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c89478-ea79-4ad5-acdd-034b90b701bc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:54.223531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c89478-ea79-4ad5-acdd-034b90b701bc None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:54.225193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 157/300] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:54.230130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c89478-ea79-4ad5-acdd-034b90b701bc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:54.289338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff7642-d19b-4e70-a3d0-c1efe683979f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:54.289928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:54.325113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ce507-0e04-40ae-957d-799c19307ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:54.325747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ce507-0e04-40ae-957d-799c19307ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:54.326020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ce507-0e04-40ae-957d-799c19307ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:54.326282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ce507-0e04-40ae-957d-799c19307ed5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:54.327448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ce507-0e04-40ae-957d-799c19307ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:54.329064 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ce507-0e04-40ae-957d-799c19307ed5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:54.330702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 144/301] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:54.331926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49e60a-09ae-414e-89a4-a8531d3d9275 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:54.332198 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49e60a-09ae-414e-89a4-a8531d3d9275 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:54.332560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49e60a-09ae-414e-89a4-a8531d3d9275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:54.348415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c89478-ea79-4ad5-acdd-034b90b701bc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:54.351664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6e49e60a-09ae-414e-89a4-a8531d3d9275 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:54.353507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6e49e60a-09ae-414e-89a4-a8531d3d9275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:54.378369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:54.387682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102811b-ee45-4932-91d6-2765873d2d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:54.388292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102811b-ee45-4932-91d6-2765873d2d79 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:54.388597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102811b-ee45-4932-91d6-2765873d2d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:54.388868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102811b-ee45-4932-91d6-2765873d2d79 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:54.389534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102811b-ee45-4932-91d6-2765873d2d79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:54.389831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102811b-ee45-4932-91d6-2765873d2d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:54.391199 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102811b-ee45-4932-91d6-2765873d2d79 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:54.403456 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 158/302] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:54.404115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102811b-ee45-4932-91d6-2765873d2d79 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:54.437050 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 145/303] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:54.437725 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49e60a-09ae-414e-89a4-a8531d3d9275 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:54.520348 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102811b-ee45-4932-91d6-2765873d2d79 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:54.537361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:54.538328 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49e60a-09ae-414e-89a4-a8531d3d9275 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:54.561933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4159b-12a0-41e4-b177-49981c661503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:54.562967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4159b-12a0-41e4-b177-49981c661503 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:54.563601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4159b-12a0-41e4-b177-49981c661503 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:54.564019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4159b-12a0-41e4-b177-49981c661503 None admin] PATH_INFO: `/v3/groups` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:54.565231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4159b-12a0-41e4-b177-49981c661503 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:54.567094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4159b-12a0-41e4-b177-49981c661503 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:54.572282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5874a3-fcb9-45cb-b341-f7687df863ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:54.572950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5874a3-fcb9-45cb-b341-f7687df863ab None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:54.573226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5874a3-fcb9-45cb-b341-f7687df863ab None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:54.573495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5874a3-fcb9-45cb-b341-f7687df863ab None admin] PATH_INFO: `/v3/users/615e87f6559c4978a59d6266889f7b16` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:54.586702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5874a3-fcb9-45cb-b341-f7687df863ab None admin] RBAC: Authorizing `identity:get_user(user_id=615e87f6559c4978a59d6266889f7b16)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:54.587472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 159/304] 10.222.0.17 () {60 vars in 1134 bytes} [Fri Sep 30 19:15:54 2022] POST /identity/v3/groups => generated 229 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:54.588214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5874a3-fcb9-45cb-b341-f7687df863ab None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:54.589029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13a3ea-f2cb-4c57-a204-dff9530ee13f None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:54.589464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13a3ea-f2cb-4c57-a204-dff9530ee13f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:54.589948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13a3ea-f2cb-4c57-a204-dff9530ee13f None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:54.590235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 146/305] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:54 2022] GET /identity/v3/users/615e87f6559c4978a59d6266889f7b16 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:54.591508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 160/306] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:54.597836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d54ea-04b9-4ce7-b66a-49c3a9bb1106 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:54.598186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d54ea-04b9-4ce7-b66a-49c3a9bb1106 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:54.598467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d54ea-04b9-4ce7-b66a-49c3a9bb1106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:54.599208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13a3ea-f2cb-4c57-a204-dff9530ee13f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:54.617936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ee3d54ea-04b9-4ce7-b66a-49c3a9bb1106 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:54.619321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ee3d54ea-04b9-4ce7-b66a-49c3a9bb1106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:54.632930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:54.711193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 147/307] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:54.712841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d54ea-04b9-4ce7-b66a-49c3a9bb1106 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:54.739718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13a3ea-f2cb-4c57-a204-dff9530ee13f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:54.784954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fba83-d313-42ee-9110-b7fef60d1a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:54.785576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fba83-d313-42ee-9110-b7fef60d1a07 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:54.785850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fba83-d313-42ee-9110-b7fef60d1a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:54.786129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fba83-d313-42ee-9110-b7fef60d1a07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:54.787377 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fba83-d313-42ee-9110-b7fef60d1a07 None admin] RBAC: Adding query filter params (role.id=fe9a4b7f28a04337bb3889d5fd2cf17d, user.id=615e87f6559c4978a59d6266889f7b16, scope.system=all) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:54.788099 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fba83-d313-42ee-9110-b7fef60d1a07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:54.789576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fba83-d313-42ee-9110-b7fef60d1a07 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:54.795446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 161/308] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 30 19:15:54 2022] GET /identity/v3/role_assignments?user.id=615e87f6559c4978a59d6266889f7b16&scope.system=all&role.id=fe9a4b7f28a04337bb3889d5fd2cf17d => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:15:54.842722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d54ea-04b9-4ce7-b66a-49c3a9bb1106 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:54.881261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1cb53-7ba4-4ff7-8206-bb109b0f946c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:54.882311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1cb53-7ba4-4ff7-8206-bb109b0f946c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:54.882789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1cb53-7ba4-4ff7-8206-bb109b0f946c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:54.883276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1cb53-7ba4-4ff7-8206-bb109b0f946c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:54.895669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1cb53-7ba4-4ff7-8206-bb109b0f946c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5ff1cb53-7ba4-4ff7-8206-bb109b0f946c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:54.901689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:54.902636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:54.903453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:54.906104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 148/309] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:54.913575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fba83-d313-42ee-9110-b7fef60d1a07 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:54.958870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:55.003995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1cb53-7ba4-4ff7-8206-bb109b0f946c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:55.016709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:55.027896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fba83-d313-42ee-9110-b7fef60d1a07 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:55.068522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ad2c6-926e-4a81-b6ea-8556ccd368ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:55.069190 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad2c6-926e-4a81-b6ea-8556ccd368ac None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:55.069394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad2c6-926e-4a81-b6ea-8556ccd368ac None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:55.069663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad2c6-926e-4a81-b6ea-8556ccd368ac None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:55.070319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ad2c6-926e-4a81-b6ea-8556ccd368ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:55.070611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ad2c6-926e-4a81-b6ea-8556ccd368ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:55.072683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ad2c6-926e-4a81-b6ea-8556ccd368ac None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:55.082102 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 162/310] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:55.087298 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ad2c6-926e-4a81-b6ea-8556ccd368ac None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:55.123581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1cb53-7ba4-4ff7-8206-bb109b0f946c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:55.162274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea3c21-b0e7-43a4-891f-662ee4ca65b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:55.162956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea3c21-b0e7-43a4-891f-662ee4ca65b4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:55.163229 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea3c21-b0e7-43a4-891f-662ee4ca65b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:55.163493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea3c21-b0e7-43a4-891f-662ee4ca65b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:55.175462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea3c21-b0e7-43a4-891f-662ee4ca65b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7bea3c21-b0e7-43a4-891f-662ee4ca65b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:55.180834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:55.181742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:55.182560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:55.186791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 149/311] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:55.187793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea3c21-b0e7-43a4-891f-662ee4ca65b4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:55.209144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ad2c6-926e-4a81-b6ea-8556ccd368ac None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:55.212050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:55.257749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13771018-b99e-47bf-a555-c8ed27588dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:55.258750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13771018-b99e-47bf-a555-c8ed27588dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:55.259676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13771018-b99e-47bf-a555-c8ed27588dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:55.260056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13771018-b99e-47bf-a555-c8ed27588dc5 None admin] PATH_INFO: `/v3/users/d316133186494183b49d933eb9c36f70` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:55.261606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13771018-b99e-47bf-a555-c8ed27588dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=d316133186494183b49d933eb9c36f70)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:55.264028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13771018-b99e-47bf-a555-c8ed27588dc5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:55.270661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 163/312] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:55 2022] GET /identity/v3/users/d316133186494183b49d933eb9c36f70 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:55.272229 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:55.273730 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13771018-b99e-47bf-a555-c8ed27588dc5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:55.314101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea3c21-b0e7-43a4-891f-662ee4ca65b4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:55.358336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2ab67-02ed-4883-936b-a47b11c0c421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:55.358951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2ab67-02ed-4883-936b-a47b11c0c421 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:55.359230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2ab67-02ed-4883-936b-a47b11c0c421 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:55.359500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2ab67-02ed-4883-936b-a47b11c0c421 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:55.360165 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2ab67-02ed-4883-936b-a47b11c0c421 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:55.360493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2ab67-02ed-4883-936b-a47b11c0c421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:55.362316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2ab67-02ed-4883-936b-a47b11c0c421 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:55.373227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 150/313] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:55.379030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2ab67-02ed-4883-936b-a47b11c0c421 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:55.395080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13771018-b99e-47bf-a555-c8ed27588dc5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:55.442498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034e3f8-e036-4465-a9f6-52e93dd8a38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:55.443487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034e3f8-e036-4465-a9f6-52e93dd8a38e None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:55.443931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034e3f8-e036-4465-a9f6-52e93dd8a38e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:55.444400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034e3f8-e036-4465-a9f6-52e93dd8a38e None admin] PATH_INFO: `/v3/system/users/d316133186494183b49d933eb9c36f70/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:55.447237 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034e3f8-e036-4465-a9f6-52e93dd8a38e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d316133186494183b49d933eb9c36f70, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:55.449301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034e3f8-e036-4465-a9f6-52e93dd8a38e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:55.469434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 164/314] 10.222.0.17 () {58 vars in 1334 bytes} [Fri Sep 30 19:15:55 2022] PUT /identity/v3/system/users/d316133186494183b49d933eb9c36f70/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:15:55.504132 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2ab67-02ed-4883-936b-a47b11c0c421 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:55.535229 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:55.543902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252f345-7ce4-4ce4-ad26-9601ad076a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:55.549326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252f345-7ce4-4ce4-ad26-9601ad076a9a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:55.549916 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252f345-7ce4-4ce4-ad26-9601ad076a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:55.550441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252f345-7ce4-4ce4-ad26-9601ad076a9a None admin] PATH_INFO: `/v3/users/2e96750ad6674a378ed9d8fab296d9bd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:55.552025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252f345-7ce4-4ce4-ad26-9601ad076a9a None admin] RBAC: Authorizing `identity:get_user(user_id=2e96750ad6674a378ed9d8fab296d9bd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:55.553745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252f345-7ce4-4ce4-ad26-9601ad076a9a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:55.556797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 151/315] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:55 2022] GET /identity/v3/users/2e96750ad6674a378ed9d8fab296d9bd => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:55.562349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034e3f8-e036-4465-a9f6-52e93dd8a38e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:55.600519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:55.683246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034e3f8-e036-4465-a9f6-52e93dd8a38e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:55.721680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d4281-5b04-4967-9377-4ae627d579ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:55.722657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d4281-5b04-4967-9377-4ae627d579ed None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:55.723216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d4281-5b04-4967-9377-4ae627d579ed None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:55.723719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d4281-5b04-4967-9377-4ae627d579ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:55.795043 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d4281-5b04-4967-9377-4ae627d579ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:55.796466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d4281-5b04-4967-9377-4ae627d579ed None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:55.827885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:55.897609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 165/316] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:15:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:15:55.903474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252f345-7ce4-4ce4-ad26-9601ad076a9a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:56.013924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252f345-7ce4-4ce4-ad26-9601ad076a9a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:56.049588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd500d-4f75-428a-bca4-070fcaac9021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:56.050135 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd500d-4f75-428a-bca4-070fcaac9021 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:56.050410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd500d-4f75-428a-bca4-070fcaac9021 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:56.050677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd500d-4f75-428a-bca4-070fcaac9021 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:56.051790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd500d-4f75-428a-bca4-070fcaac9021 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:56.053323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd500d-4f75-428a-bca4-070fcaac9021 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:56.055301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 152/317] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:15:55 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:56.061112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d4281-5b04-4967-9377-4ae627d579ed None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:56.160607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d4281-5b04-4967-9377-4ae627d579ed None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:56.192470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:56.195791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ab3d7-4243-44d3-a096-20631901b5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:56.197145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ab3d7-4243-44d3-a096-20631901b5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:56.197492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ab3d7-4243-44d3-a096-20631901b5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:56.197832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ab3d7-4243-44d3-a096-20631901b5b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:56.198914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ab3d7-4243-44d3-a096-20631901b5b8 None admin] RBAC: Adding query filter params (role.id=fe9a4b7f28a04337bb3889d5fd2cf17d, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f, user.id=2e96750ad6674a378ed9d8fab296d9bd) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:56.199565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ab3d7-4243-44d3-a096-20631901b5b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:56.201376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ab3d7-4243-44d3-a096-20631901b5b8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:56.212506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 166/318] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:15:56 2022] GET /identity/v3/role_assignments?user.id=2e96750ad6674a378ed9d8fab296d9bd&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=fe9a4b7f28a04337bb3889d5fd2cf17d => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:56.871509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad2bcf-b237-4c87-aea3-3efc55f36b5b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:56.871882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad2bcf-b237-4c87-aea3-3efc55f36b5b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:56.872057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad2bcf-b237-4c87-aea3-3efc55f36b5b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:56.873552 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 153/319] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:56.878479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee915b-34fd-4161-85fe-3ea81d80cfeb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:56.878903 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee915b-34fd-4161-85fe-3ea81d80cfeb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:56.879056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee915b-34fd-4161-85fe-3ea81d80cfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:56.898021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-dcee915b-34fd-4161-85fe-3ea81d80cfeb None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:56.899243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dcee915b-34fd-4161-85fe-3ea81d80cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:56.969723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 167/320] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:57.020396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d60f1a-b5e8-4a13-8e83-516844e9b48d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:57.020887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d60f1a-b5e8-4a13-8e83-516844e9b48d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:57.020962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d60f1a-b5e8-4a13-8e83-516844e9b48d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:57.022441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 154/321] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:57.027871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c048f5-5384-4506-a155-ff9b71446d99 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:57.028164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c048f5-5384-4506-a155-ff9b71446d99 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:57.028475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c048f5-5384-4506-a155-ff9b71446d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:57.046854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f3c048f5-5384-4506-a155-ff9b71446d99 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:57.048020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f3c048f5-5384-4506-a155-ff9b71446d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:57.114489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 168/322] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:15:57.232036 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d60f1a-b5e8-4a13-8e83-516844e9b48d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:57.296086 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:57.334612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d60f1a-b5e8-4a13-8e83-516844e9b48d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:57.368634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f57c7-cc14-4e43-974e-782ca19132af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:57.369263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f57c7-cc14-4e43-974e-782ca19132af None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:57.369534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f57c7-cc14-4e43-974e-782ca19132af None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:57.369795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f57c7-cc14-4e43-974e-782ca19132af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:57.370865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f57c7-cc14-4e43-974e-782ca19132af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:57.372283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f57c7-cc14-4e43-974e-782ca19132af None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:57.376504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 155/323] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:15:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:57.379253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c048f5-5384-4506-a155-ff9b71446d99 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:57.397643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f57c7-cc14-4e43-974e-782ca19132af None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:57.439349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:57.486678 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c048f5-5384-4506-a155-ff9b71446d99 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:57.506760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f57c7-cc14-4e43-974e-782ca19132af None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:57.522005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b51a27-d3be-45aa-bdf7-d8fb1e66137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:57.522637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b51a27-d3be-45aa-bdf7-d8fb1e66137e None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:57.522914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b51a27-d3be-45aa-bdf7-d8fb1e66137e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:57.523180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b51a27-d3be-45aa-bdf7-d8fb1e66137e None admin] PATH_INFO: `/v3/groups` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:57.523811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b51a27-d3be-45aa-bdf7-d8fb1e66137e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:57.525295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b51a27-d3be-45aa-bdf7-d8fb1e66137e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:57.538086 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:57.552406 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 169/324] 10.222.0.17 () {60 vars in 1134 bytes} [Fri Sep 30 19:15:57 2022] POST /identity/v3/groups => generated 226 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:15:57.552617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d69fc-5e42-4477-b959-75adfea7fe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:57.552617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d69fc-5e42-4477-b959-75adfea7fe76 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:57.552617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d69fc-5e42-4477-b959-75adfea7fe76 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:57.552617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d69fc-5e42-4477-b959-75adfea7fe76 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:57.564109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d69fc-5e42-4477-b959-75adfea7fe76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9f9d69fc-5e42-4477-b959-75adfea7fe76 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:57.569569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:57.570557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:15:57.572410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 156/325] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:15:57.577278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b51a27-d3be-45aa-bdf7-d8fb1e66137e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:57.686482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b51a27-d3be-45aa-bdf7-d8fb1e66137e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:57.704292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:57.724904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617a86f-b47b-4c8b-b539-fea3b9d14079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:57.725512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617a86f-b47b-4c8b-b539-fea3b9d14079 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:57.725783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617a86f-b47b-4c8b-b539-fea3b9d14079 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:57.726045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617a86f-b47b-4c8b-b539-fea3b9d14079 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:57.726702 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617a86f-b47b-4c8b-b539-fea3b9d14079 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:57.726993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617a86f-b47b-4c8b-b539-fea3b9d14079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:57.732563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617a86f-b47b-4c8b-b539-fea3b9d14079 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:57.745965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 170/326] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:57.747304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3b2db-3673-4e6a-968c-3c04643b8d88 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:57.747584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3b2db-3673-4e6a-968c-3c04643b8d88 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:57.747851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3b2db-3673-4e6a-968c-3c04643b8d88 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:57.749324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 157/327] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:15:57.751847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617a86f-b47b-4c8b-b539-fea3b9d14079 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:57.754441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d11b1d-69a5-4ca2-bb5e-8d2a70d87150 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:57.754724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d11b1d-69a5-4ca2-bb5e-8d2a70d87150 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:57.755035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d11b1d-69a5-4ca2-bb5e-8d2a70d87150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:57.785383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-94d11b1d-69a5-4ca2-bb5e-8d2a70d87150 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:15:57.786769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-94d11b1d-69a5-4ca2-bb5e-8d2a70d87150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:15:57.912397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 158/328] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:15:57.914482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617a86f-b47b-4c8b-b539-fea3b9d14079 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:57.949991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcc8ba-0eb6-40d8-825e-2160518b66b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:57.950626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcc8ba-0eb6-40d8-825e-2160518b66b5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:57.950906 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcc8ba-0eb6-40d8-825e-2160518b66b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:57.951179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcc8ba-0eb6-40d8-825e-2160518b66b5 None admin] PATH_INFO: `/v3/users/615e87f6559c4978a59d6266889f7b16` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:57.952319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcc8ba-0eb6-40d8-825e-2160518b66b5 None admin] RBAC: Authorizing `identity:get_user(user_id=615e87f6559c4978a59d6266889f7b16)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:57.953776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcc8ba-0eb6-40d8-825e-2160518b66b5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:57.955897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 171/329] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:57 2022] GET /identity/v3/users/615e87f6559c4978a59d6266889f7b16 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:57.961122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d11b1d-69a5-4ca2-bb5e-8d2a70d87150 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:58.069506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d11b1d-69a5-4ca2-bb5e-8d2a70d87150 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:58.096689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:58.112677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c64e87-81e2-4da9-ad4c-4e583337b462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:58.113535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c64e87-81e2-4da9-ad4c-4e583337b462 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:58.113980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c64e87-81e2-4da9-ad4c-4e583337b462 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:58.114258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c64e87-81e2-4da9-ad4c-4e583337b462 None admin] PATH_INFO: `/v3/system/users/615e87f6559c4978a59d6266889f7b16/roles/fe9a4b7f28a04337bb3889d5fd2cf17d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:58.115701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c64e87-81e2-4da9-ad4c-4e583337b462 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=615e87f6559c4978a59d6266889f7b16, role_id=fe9a4b7f28a04337bb3889d5fd2cf17d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:58.117108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c64e87-81e2-4da9-ad4c-4e583337b462 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:58.135752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 159/330] 10.222.0.17 () {58 vars in 1334 bytes} [Fri Sep 30 19:15:57 2022] PUT /identity/v3/system/users/615e87f6559c4978a59d6266889f7b16/roles/fe9a4b7f28a04337bb3889d5fd2cf17d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:15:58.198449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcc8ba-0eb6-40d8-825e-2160518b66b5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:58.257116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:58.329582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcc8ba-0eb6-40d8-825e-2160518b66b5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:58.365317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77d633-711b-4b4a-8d94-5dc4d4ca9ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:58.365959 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77d633-711b-4b4a-8d94-5dc4d4ca9ffa None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:58.366242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77d633-711b-4b4a-8d94-5dc4d4ca9ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:58.366514 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77d633-711b-4b4a-8d94-5dc4d4ca9ffa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:58.378960 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77d633-711b-4b4a-8d94-5dc4d4ca9ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ea77d633-711b-4b4a-8d94-5dc4d4ca9ffa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:15:58.384980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:15:58.385881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:15:58.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 172/331] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:15:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:15:58.393654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c64e87-81e2-4da9-ad4c-4e583337b462 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:58.511744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c64e87-81e2-4da9-ad4c-4e583337b462 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:58.550196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65257b-4462-4f94-a4a5-32c68bb52b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:58.551163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65257b-4462-4f94-a4a5-32c68bb52b44 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:58.551632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65257b-4462-4f94-a4a5-32c68bb52b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:58.552088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65257b-4462-4f94-a4a5-32c68bb52b44 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:58.555356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65257b-4462-4f94-a4a5-32c68bb52b44 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:58.556459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65257b-4462-4f94-a4a5-32c68bb52b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:58.558050 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65257b-4462-4f94-a4a5-32c68bb52b44 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:58.567270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 160/332] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:15:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:58.573371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77d633-711b-4b4a-8d94-5dc4d4ca9ffa None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:58.679473 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77d633-711b-4b4a-8d94-5dc4d4ca9ffa None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:58.688814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:58.723257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f63c42-5d04-4862-80c4-7fc2c41e9ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:58.724305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f63c42-5d04-4862-80c4-7fc2c41e9ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:58.724798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f63c42-5d04-4862-80c4-7fc2c41e9ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:58.725306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f63c42-5d04-4862-80c4-7fc2c41e9ec6 None admin] PATH_INFO: `/v3/users/d316133186494183b49d933eb9c36f70` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:58.726623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f63c42-5d04-4862-80c4-7fc2c41e9ec6 None admin] RBAC: Authorizing `identity:get_user(user_id=d316133186494183b49d933eb9c36f70)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:58.728439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f63c42-5d04-4862-80c4-7fc2c41e9ec6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:58.730305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 173/333] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:15:58 2022] GET /identity/v3/users/d316133186494183b49d933eb9c36f70 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:15:58.736282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65257b-4462-4f94-a4a5-32c68bb52b44 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:15:58.793807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:15:58.847799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65257b-4462-4f94-a4a5-32c68bb52b44 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:15:58.885168 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ab85b-fd26-4442-829f-a486a833ed89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:15:58.885772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ab85b-fd26-4442-829f-a486a833ed89 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:15:58.886043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ab85b-fd26-4442-829f-a486a833ed89 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:15:58.886308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ab85b-fd26-4442-829f-a486a833ed89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:15:58.887029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ab85b-fd26-4442-829f-a486a833ed89 None admin] RBAC: Adding query filter params (role.id=43452aaa2c9c4505b4bd3dd1b2412c65, user.id=d316133186494183b49d933eb9c36f70, scope.system=all) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:15:58.887329 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ab85b-fd26-4442-829f-a486a833ed89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:15:58.888792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ab85b-fd26-4442-829f-a486a833ed89 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:15:58.901073 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 161/334] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 30 19:15:58 2022] GET /identity/v3/role_assignments?user.id=d316133186494183b49d933eb9c36f70&scope.system=all&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 510 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:00.223680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54bced-a4c3-4e39-9939-666e03c75503 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:00.224215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54bced-a4c3-4e39-9939-666e03c75503 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:00.224215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54bced-a4c3-4e39-9939-666e03c75503 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:00.225858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 174/335] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:00.231865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436306b-6073-4a49-9965-7a177f9eb1ea None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:00.232323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436306b-6073-4a49-9965-7a177f9eb1ea None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:00.232446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436306b-6073-4a49-9965-7a177f9eb1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:00.253385 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5436306b-6073-4a49-9965-7a177f9eb1ea None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:00.254592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5436306b-6073-4a49-9965-7a177f9eb1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:00.344313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 162/336] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:00.824850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a764f-1186-435c-8cfd-2398c6908c75 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:00.825204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a764f-1186-435c-8cfd-2398c6908c75 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:00.825409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a764f-1186-435c-8cfd-2398c6908c75 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:00.826972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 175/337] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:00.832630 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5634a0-bb09-48f2-a4fc-e05c7253c5b2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:00.832916 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5634a0-bb09-48f2-a4fc-e05c7253c5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:00.833187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5634a0-bb09-48f2-a4fc-e05c7253c5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:00.857344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5c5634a0-bb09-48f2-a4fc-e05c7253c5b2 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:00.858520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5c5634a0-bb09-48f2-a4fc-e05c7253c5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:00.877628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a764f-1186-435c-8cfd-2398c6908c75 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:00.898879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:00.963501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 163/338] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:00.997908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147bd32-ee62-42e6-aa09-2e4644ca5087 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:00.998211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147bd32-ee62-42e6-aa09-2e4644ca5087 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:00.998497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147bd32-ee62-42e6-aa09-2e4644ca5087 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:01.000068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 164/339] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:01.005251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382307b-c5f4-47e3-8b45-3ea0a3cb1708 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:01.005534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382307b-c5f4-47e3-8b45-3ea0a3cb1708 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:01.005830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382307b-c5f4-47e3-8b45-3ea0a3cb1708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:01.039595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5382307b-c5f4-47e3-8b45-3ea0a3cb1708 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:01.040956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5382307b-c5f4-47e3-8b45-3ea0a3cb1708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:01.079494 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a764f-1186-435c-8cfd-2398c6908c75 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:01.145434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae6142-5c4d-4e19-9951-1772674c4a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:01.146129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae6142-5c4d-4e19-9951-1772674c4a64 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:01.146402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae6142-5c4d-4e19-9951-1772674c4a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:01.146667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae6142-5c4d-4e19-9951-1772674c4a64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:01.170248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae6142-5c4d-4e19-9951-1772674c4a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:01.170970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 165/340] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a0ae6142-5c4d-4e19-9951-1772674c4a64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:01.175351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:01.176226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:01.177134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:01.181594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 176/341] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:01.187446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382307b-c5f4-47e3-8b45-3ea0a3cb1708 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:01.332708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:01.338741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382307b-c5f4-47e3-8b45-3ea0a3cb1708 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:01.392393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48039ba-0787-4ef4-8d6a-43685d628d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:01.392393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48039ba-0787-4ef4-8d6a-43685d628d14 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:01.392393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48039ba-0787-4ef4-8d6a-43685d628d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:01.392393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48039ba-0787-4ef4-8d6a-43685d628d14 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:01.392786 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48039ba-0787-4ef4-8d6a-43685d628d14 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:01.392786 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48039ba-0787-4ef4-8d6a-43685d628d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:01.393507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48039ba-0787-4ef4-8d6a-43685d628d14 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:01.402642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 166/342] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:16:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:01.408120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae6142-5c4d-4e19-9951-1772674c4a64 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:01.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:01.544473 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae6142-5c4d-4e19-9951-1772674c4a64 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:01.581473 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840f19f-b90b-4d6b-a13a-48173b85dff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:01.582076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840f19f-b90b-4d6b-a13a-48173b85dff6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:01.582351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840f19f-b90b-4d6b-a13a-48173b85dff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:01.582617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840f19f-b90b-4d6b-a13a-48173b85dff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:01.610234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48039ba-0787-4ef4-8d6a-43685d628d14 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:01.674047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840f19f-b90b-4d6b-a13a-48173b85dff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:01.675982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840f19f-b90b-4d6b-a13a-48173b85dff6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:01.718668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:01.747455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48039ba-0787-4ef4-8d6a-43685d628d14 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:01.789902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336f38f-679b-4a94-bf75-805c49d896fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:01.790545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336f38f-679b-4a94-bf75-805c49d896fa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:01.790817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336f38f-679b-4a94-bf75-805c49d896fa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:01.791080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336f38f-679b-4a94-bf75-805c49d896fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:01.796740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 177/343] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:01.797812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840f19f-b90b-4d6b-a13a-48173b85dff6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:01.809574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336f38f-679b-4a94-bf75-805c49d896fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1336f38f-679b-4a94-bf75-805c49d896fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:01.814053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:01.814970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:01.815806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 167/344] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:01.816529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336f38f-679b-4a94-bf75-805c49d896fa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:01.837686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:01.908049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840f19f-b90b-4d6b-a13a-48173b85dff6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:01.947969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1ece0-5c8c-4152-8446-5f8f52b706f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:01.948910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1ece0-5c8c-4152-8446-5f8f52b706f0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:01.949328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1ece0-5c8c-4152-8446-5f8f52b706f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:01.949738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1ece0-5c8c-4152-8446-5f8f52b706f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:01.960249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:01.969902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1ece0-5c8c-4152-8446-5f8f52b706f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-8df1ece0-5c8c-4152-8446-5f8f52b706f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:01.975049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:01.975971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:01.976946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 178/345] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:16:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:01.977684 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1ece0-5c8c-4152-8446-5f8f52b706f0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:01.984226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336f38f-679b-4a94-bf75-805c49d896fa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:02.033020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d5f4f-d757-43eb-bdc5-b9fc77aa56d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:02.033658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d5f4f-d757-43eb-bdc5-b9fc77aa56d9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.033936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d5f4f-d757-43eb-bdc5-b9fc77aa56d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.034207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d5f4f-d757-43eb-bdc5-b9fc77aa56d9 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.036512 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d5f4f-d757-43eb-bdc5-b9fc77aa56d9 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:02.038226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d5f4f-d757-43eb-bdc5-b9fc77aa56d9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:02.040830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 168/346] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:01 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:02.041988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb54de-5c2e-497d-801c-014b9d5bdc69 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.042269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb54de-5c2e-497d-801c-014b9d5bdc69 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.042543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb54de-5c2e-497d-801c-014b9d5bdc69 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.043840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 169/347] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:02.048478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb54de-5c2e-497d-801c-014b9d5bdc69 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:02.100165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1ece0-5c8c-4152-8446-5f8f52b706f0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:02.137403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8176b-cb8d-4a3e-9639-469f48faa01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:02.138379 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8176b-cb8d-4a3e-9639-469f48faa01e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.138888 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8176b-cb8d-4a3e-9639-469f48faa01e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.139376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8176b-cb8d-4a3e-9639-469f48faa01e None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.140224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8176b-cb8d-4a3e-9639-469f48faa01e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:02.140738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8176b-cb8d-4a3e-9639-469f48faa01e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:02.142310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8176b-cb8d-4a3e-9639-469f48faa01e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:02.148790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:02.151671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 179/348] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:16:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:02.152750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8176b-cb8d-4a3e-9639-469f48faa01e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:02.159405 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb54de-5c2e-497d-801c-014b9d5bdc69 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:02.195613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedc3c6-fb87-41f3-9c1a-e8b27b87e692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:02.196794 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedc3c6-fb87-41f3-9c1a-e8b27b87e692 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.197075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedc3c6-fb87-41f3-9c1a-e8b27b87e692 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.197347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedc3c6-fb87-41f3-9c1a-e8b27b87e692 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.198025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedc3c6-fb87-41f3-9c1a-e8b27b87e692 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:02.198327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedc3c6-fb87-41f3-9c1a-e8b27b87e692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:02.199880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcedc3c6-fb87-41f3-9c1a-e8b27b87e692 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:02.208419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 170/349] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:02.209942 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19925a39-7382-48be-9319-44930a688552 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.210330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19925a39-7382-48be-9319-44930a688552 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.210703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19925a39-7382-48be-9319-44930a688552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.218571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:02.234514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-19925a39-7382-48be-9319-44930a688552 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:02.235841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-19925a39-7382-48be-9319-44930a688552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:02.272421 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8176b-cb8d-4a3e-9639-469f48faa01e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:02.315723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70c5ea-0cd8-45d7-bb49-2533caf54421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:02.316689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70c5ea-0cd8-45d7-bb49-2533caf54421 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.317224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70c5ea-0cd8-45d7-bb49-2533caf54421 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.317648 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70c5ea-0cd8-45d7-bb49-2533caf54421 None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.319076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 171/350] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:02.320030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19925a39-7382-48be-9319-44930a688552 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:02.325093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70c5ea-0cd8-45d7-bb49-2533caf54421 None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:02.326752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70c5ea-0cd8-45d7-bb49-2533caf54421 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:02.328938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 180/351] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:02 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:02.329869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70c5ea-0cd8-45d7-bb49-2533caf54421 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:02.451809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:02.457168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70c5ea-0cd8-45d7-bb49-2533caf54421 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:02.478652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19925a39-7382-48be-9319-44930a688552 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:02.482097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:02.501035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd5a1d-32e3-41fe-ae0e-e6a5f6bbd1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:02.501807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd5a1d-32e3-41fe-ae0e-e6a5f6bbd1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.502092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd5a1d-32e3-41fe-ae0e-e6a5f6bbd1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.502365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd5a1d-32e3-41fe-ae0e-e6a5f6bbd1b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.522012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5731ba-2b81-4be0-9e60-5513dfe3e65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:02.522961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5731ba-2b81-4be0-9e60-5513dfe3e65c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.523435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5731ba-2b81-4be0-9e60-5513dfe3e65c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.524030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5731ba-2b81-4be0-9e60-5513dfe3e65c None admin] PATH_INFO: `/v3/users/615e87f6559c4978a59d6266889f7b16` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.526150 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5731ba-2b81-4be0-9e60-5513dfe3e65c None admin] RBAC: Authorizing `identity:get_user(user_id=615e87f6559c4978a59d6266889f7b16)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:02.528242 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5731ba-2b81-4be0-9e60-5513dfe3e65c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:02.535994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 172/352] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:16:02 2022] GET /identity/v3/users/615e87f6559c4978a59d6266889f7b16 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:02.537483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5731ba-2b81-4be0-9e60-5513dfe3e65c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:02.582718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd5a1d-32e3-41fe-ae0e-e6a5f6bbd1b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:02.584576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd5a1d-32e3-41fe-ae0e-e6a5f6bbd1b8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:02.648529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5731ba-2b81-4be0-9e60-5513dfe3e65c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:02.686833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03faae-fc07-4ed0-b0b6-87db287393e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:02.687905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03faae-fc07-4ed0-b0b6-87db287393e9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.688392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03faae-fc07-4ed0-b0b6-87db287393e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.688888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03faae-fc07-4ed0-b0b6-87db287393e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.689803 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03faae-fc07-4ed0-b0b6-87db287393e9 None admin] RBAC: Adding query filter params (group.id=b0ade2d70c52410fba43d25bd173c0d4, role.id=43452aaa2c9c4505b4bd3dd1b2412c65, scope.project.id=63a660324ea34aa886a24836744d158b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:02.690292 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03faae-fc07-4ed0-b0b6-87db287393e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:02.692176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03faae-fc07-4ed0-b0b6-87db287393e9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:02.698734 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 173/353] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 30 19:16:02 2022] GET /identity/v3/role_assignments?group.id=b0ade2d70c52410fba43d25bd173c0d4&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:02.700042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f81d8-57ba-43c5-aabc-ff972fed7cb6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.700330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f81d8-57ba-43c5-aabc-ff972fed7cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.700644 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f81d8-57ba-43c5-aabc-ff972fed7cb6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.701900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 174/354] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:02.705501 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 181/355] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:02.706375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd5a1d-32e3-41fe-ae0e-e6a5f6bbd1b8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:02.706556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f81d8-57ba-43c5-aabc-ff972fed7cb6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:02.725891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:02.734828 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:02.836435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd5a1d-32e3-41fe-ae0e-e6a5f6bbd1b8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:02.846604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f81d8-57ba-43c5-aabc-ff972fed7cb6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:02.902566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66e2d1-488b-4c30-9593-9d954ba771f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:02.903218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66e2d1-488b-4c30-9593-9d954ba771f4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.903494 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66e2d1-488b-4c30-9593-9d954ba771f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.903767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66e2d1-488b-4c30-9593-9d954ba771f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.910847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55bb6d-81b1-4b7f-bdee-529e1dc5c6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:02.911736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55bb6d-81b1-4b7f-bdee-529e1dc5c6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.912335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55bb6d-81b1-4b7f-bdee-529e1dc5c6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.912705 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55bb6d-81b1-4b7f-bdee-529e1dc5c6b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.913511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55bb6d-81b1-4b7f-bdee-529e1dc5c6b3 None admin] RBAC: Adding query filter params (role.id=fe9a4b7f28a04337bb3889d5fd2cf17d, user.id=615e87f6559c4978a59d6266889f7b16, scope.system=all) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:02.913891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55bb6d-81b1-4b7f-bdee-529e1dc5c6b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:02.915930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55bb6d-81b1-4b7f-bdee-529e1dc5c6b3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:02.919301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66e2d1-488b-4c30-9593-9d954ba771f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-fa66e2d1-488b-4c30-9593-9d954ba771f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:02.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:02.925592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:02.926445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 182/356] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:02.927633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060abdf4-2042-403f-adf3-003d71eb9920 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:02.927633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060abdf4-2042-403f-adf3-003d71eb9920 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:02.927793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060abdf4-2042-403f-adf3-003d71eb9920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:02.937734 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 175/357] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Sep 30 19:16:02 2022] GET /identity/v3/role_assignments?user.id=615e87f6559c4978a59d6266889f7b16&scope.system=all&role.id=fe9a4b7f28a04337bb3889d5fd2cf17d => generated 510 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:02.939157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55bb6d-81b1-4b7f-bdee-529e1dc5c6b3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:02.954108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-060abdf4-2042-403f-adf3-003d71eb9920 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:02.955311 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-060abdf4-2042-403f-adf3-003d71eb9920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:03.031238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:03.046448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 183/358] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:03.047696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060abdf4-2042-403f-adf3-003d71eb9920 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:03.060803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:03.074709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55bb6d-81b1-4b7f-bdee-529e1dc5c6b3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:03.117634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bda9b1-6e35-4204-a28e-39ac16efbcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:03.118722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bda9b1-6e35-4204-a28e-39ac16efbcb2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:03.119235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bda9b1-6e35-4204-a28e-39ac16efbcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:03.119713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bda9b1-6e35-4204-a28e-39ac16efbcb2 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:03.121292 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bda9b1-6e35-4204-a28e-39ac16efbcb2 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:03.123943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bda9b1-6e35-4204-a28e-39ac16efbcb2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:03.128119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 176/359] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:02 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:03.131137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6ab26-9073-4c29-90a8-1595d019d468 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:03.134448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6ab26-9073-4c29-90a8-1595d019d468 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:03.135972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6ab26-9073-4c29-90a8-1595d019d468 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:03.138654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 177/360] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:03.139890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6ab26-9073-4c29-90a8-1595d019d468 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:03.202587 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060abdf4-2042-403f-adf3-003d71eb9920 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:03.273880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec54b7-8169-41f9-a8d0-42ebd464509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:03.274986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec54b7-8169-41f9-a8d0-42ebd464509b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:03.275265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec54b7-8169-41f9-a8d0-42ebd464509b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:03.275536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec54b7-8169-41f9-a8d0-42ebd464509b None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:03.279072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec54b7-8169-41f9-a8d0-42ebd464509b None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:03.279384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec54b7-8169-41f9-a8d0-42ebd464509b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:03.280906 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec54b7-8169-41f9-a8d0-42ebd464509b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:03.302644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 184/361] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:16:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:03.304126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de22195-4ef3-4f4e-99e4-7d878a1d26fe None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:03.304406 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de22195-4ef3-4f4e-99e4-7d878a1d26fe None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:03.304681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de22195-4ef3-4f4e-99e4-7d878a1d26fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:03.322548 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6ab26-9073-4c29-90a8-1595d019d468 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:03.341689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2de22195-4ef3-4f4e-99e4-7d878a1d26fe None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:03.343272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2de22195-4ef3-4f4e-99e4-7d878a1d26fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:03.362063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1bf29-f04b-4fb5-9f84-7942e8c3ef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:03.362709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1bf29-f04b-4fb5-9f84-7942e8c3ef02 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:03.362987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1bf29-f04b-4fb5-9f84-7942e8c3ef02 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:03.363352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1bf29-f04b-4fb5-9f84-7942e8c3ef02 None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:03.364554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1bf29-f04b-4fb5-9f84-7942e8c3ef02 None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:03.366091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1bf29-f04b-4fb5-9f84-7942e8c3ef02 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:03.368156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 178/362] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:03 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:03.368955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1bf29-f04b-4fb5-9f84-7942e8c3ef02 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:03.372697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:03.448857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 185/363] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:03.449795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de22195-4ef3-4f4e-99e4-7d878a1d26fe None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:03.452306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:03.539474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1bf29-f04b-4fb5-9f84-7942e8c3ef02 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:03.593076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180e8ea-26cb-4007-934d-a32ca8f37cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:03.593927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180e8ea-26cb-4007-934d-a32ca8f37cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:03.594646 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180e8ea-26cb-4007-934d-a32ca8f37cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:03.595076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180e8ea-26cb-4007-934d-a32ca8f37cc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:03.602549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de22195-4ef3-4f4e-99e4-7d878a1d26fe None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:03.638523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edb0d2-3383-4bd8-ba18-77c9c11e0974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:03.639330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edb0d2-3383-4bd8-ba18-77c9c11e0974 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:03.639748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edb0d2-3383-4bd8-ba18-77c9c11e0974 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:03.640109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edb0d2-3383-4bd8-ba18-77c9c11e0974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:03.640972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2edb0d2-3383-4bd8-ba18-77c9c11e0974 None admin] RBAC: Adding query filter params (group.id=b0ade2d70c52410fba43d25bd173c0d4, role.id=9c17da6cc0e54b67af7cb7fb68638a7e, scope.project.id=63a660324ea34aa886a24836744d158b) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:03.641560 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2edb0d2-3383-4bd8-ba18-77c9c11e0974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:03.643024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2edb0d2-3383-4bd8-ba18-77c9c11e0974 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:03.652789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 186/364] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 30 19:16:03 2022] GET /identity/v3/role_assignments?group.id=b0ade2d70c52410fba43d25bd173c0d4&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=9c17da6cc0e54b67af7cb7fb68638a7e => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:03.652789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edb0d2-3383-4bd8-ba18-77c9c11e0974 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:03.682484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180e8ea-26cb-4007-934d-a32ca8f37cc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:03.688844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180e8ea-26cb-4007-934d-a32ca8f37cc7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:03.692354 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:03.728382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:03.766652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edb0d2-3383-4bd8-ba18-77c9c11e0974 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:03.809240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ed245-baa7-479e-8a84-e6010130b87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:03.809947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ed245-baa7-479e-8a84-e6010130b87e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:03.810278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ed245-baa7-479e-8a84-e6010130b87e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:03.810611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ed245-baa7-479e-8a84-e6010130b87e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:03.822490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ed245-baa7-479e-8a84-e6010130b87e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-df3ed245-baa7-479e-8a84-e6010130b87e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:03.827984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:03.828940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 187/365] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:16:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:03.830180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ed245-baa7-479e-8a84-e6010130b87e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:03.859201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 179/366] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:03.860856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180e8ea-26cb-4007-934d-a32ca8f37cc7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:03.955383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ed245-baa7-479e-8a84-e6010130b87e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:03.985966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:03.994883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05d67b-f39e-402b-adde-9a79a05255bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:03.995518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05d67b-f39e-402b-adde-9a79a05255bd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:03.995793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05d67b-f39e-402b-adde-9a79a05255bd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:03.996061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05d67b-f39e-402b-adde-9a79a05255bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:04.006670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05d67b-f39e-402b-adde-9a79a05255bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:04.010872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180e8ea-26cb-4007-934d-a32ca8f37cc7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-4a05d67b-f39e-402b-adde-9a79a05255bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:04.012059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:04.013001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:04.013814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 188/367] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:04.014491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05d67b-f39e-402b-adde-9a79a05255bd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:04.016438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:04.058151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d777b-f538-4e99-b609-b426e2ba0abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:04.059650 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d777b-f538-4e99-b609-b426e2ba0abd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:04.059893 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d777b-f538-4e99-b609-b426e2ba0abd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:04.060139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d777b-f538-4e99-b609-b426e2ba0abd None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:04.060832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d777b-f538-4e99-b609-b426e2ba0abd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:04.061129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d777b-f538-4e99-b609-b426e2ba0abd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:04.062518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d777b-f538-4e99-b609-b426e2ba0abd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:04.074675 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 180/368] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:04.075487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d777b-f538-4e99-b609-b426e2ba0abd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:04.142391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05d67b-f39e-402b-adde-9a79a05255bd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:04.183070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a634dc6-0d76-4595-a3bd-1f5a8c47a72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:04.184085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a634dc6-0d76-4595-a3bd-1f5a8c47a72d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:04.184609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a634dc6-0d76-4595-a3bd-1f5a8c47a72d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:04.185325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a634dc6-0d76-4595-a3bd-1f5a8c47a72d None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:04.187106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a634dc6-0d76-4595-a3bd-1f5a8c47a72d None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:04.189537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a634dc6-0d76-4595-a3bd-1f5a8c47a72d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:04.196793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 189/369] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:04 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:04.197773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a634dc6-0d76-4595-a3bd-1f5a8c47a72d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:04.203419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d777b-f538-4e99-b609-b426e2ba0abd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:04.248411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb6993-f812-4c26-9c5c-bf6f6d51f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:04.249419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6993-f812-4c26-9c5c-bf6f6d51f8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:04.249927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6993-f812-4c26-9c5c-bf6f6d51f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:04.250391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6993-f812-4c26-9c5c-bf6f6d51f8f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:04.251251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb6993-f812-4c26-9c5c-bf6f6d51f8f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:04.251804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb6993-f812-4c26-9c5c-bf6f6d51f8f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:04.253532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb6993-f812-4c26-9c5c-bf6f6d51f8f5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:04.263239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 181/370] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:04.264528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb6993-f812-4c26-9c5c-bf6f6d51f8f5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:04.269954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:04.300462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:04.339273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a634dc6-0d76-4595-a3bd-1f5a8c47a72d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:04.386058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523213b-d971-4e64-aebe-3f8d826e282a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:04.387057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523213b-d971-4e64-aebe-3f8d826e282a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:04.387544 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523213b-d971-4e64-aebe-3f8d826e282a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:04.388024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523213b-d971-4e64-aebe-3f8d826e282a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:04.408897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb6993-f812-4c26-9c5c-bf6f6d51f8f5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:04.457260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41b69f-8036-415a-9258-2b9d0321275b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:04.457896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41b69f-8036-415a-9258-2b9d0321275b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:04.458372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41b69f-8036-415a-9258-2b9d0321275b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:04.458642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41b69f-8036-415a-9258-2b9d0321275b None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:04.460069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41b69f-8036-415a-9258-2b9d0321275b None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:04.461600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41b69f-8036-415a-9258-2b9d0321275b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:04.468470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 182/371] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:04 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:04.468470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41b69f-8036-415a-9258-2b9d0321275b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:04.488400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523213b-d971-4e64-aebe-3f8d826e282a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:04.488913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523213b-d971-4e64-aebe-3f8d826e282a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:04.552402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:04.580653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:04.586917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41b69f-8036-415a-9258-2b9d0321275b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:04.606986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 190/372] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:04.608666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523213b-d971-4e64-aebe-3f8d826e282a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:04.631607 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06b5ed-3ef7-42a5-a42d-daf1a31af8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:04.632228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b5ed-3ef7-42a5-a42d-daf1a31af8ee None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:04.632536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b5ed-3ef7-42a5-a42d-daf1a31af8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:04.632807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b5ed-3ef7-42a5-a42d-daf1a31af8ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:04.707149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06b5ed-3ef7-42a5-a42d-daf1a31af8ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:04.708630 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06b5ed-3ef7-42a5-a42d-daf1a31af8ee None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:04.726967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523213b-d971-4e64-aebe-3f8d826e282a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:04.767282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e92062-932a-4440-aa57-96545c0adc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:04.768289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e92062-932a-4440-aa57-96545c0adc21 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:04.768853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e92062-932a-4440-aa57-96545c0adc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:04.769469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e92062-932a-4440-aa57-96545c0adc21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:04.770390 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e92062-932a-4440-aa57-96545c0adc21 None admin] RBAC: Adding query filter params (group.id=b0ade2d70c52410fba43d25bd173c0d4, role.id=43452aaa2c9c4505b4bd3dd1b2412c65, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:04.770921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e92062-932a-4440-aa57-96545c0adc21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:04.773285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e92062-932a-4440-aa57-96545c0adc21 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:04.780331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 191/373] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 30 19:16:04 2022] GET /identity/v3/role_assignments?group.id=b0ade2d70c52410fba43d25bd173c0d4&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:04.781521 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e92062-932a-4440-aa57-96545c0adc21 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:04.832814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:04.861003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:04.871618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 183/374] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:04.877299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06b5ed-3ef7-42a5-a42d-daf1a31af8ee None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:05.013025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06b5ed-3ef7-42a5-a42d-daf1a31af8ee None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:05.040795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e92062-932a-4440-aa57-96545c0adc21 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:05.062804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7d39a-2d48-4577-9f8b-1967b627ab81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:05.063580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7d39a-2d48-4577-9f8b-1967b627ab81 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:05.063856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7d39a-2d48-4577-9f8b-1967b627ab81 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:05.064234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7d39a-2d48-4577-9f8b-1967b627ab81 None admin] PATH_INFO: `/v3/projects/e574e273dabe4a3eab67dc95d42fcb6b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:05.065394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7d39a-2d48-4577-9f8b-1967b627ab81 None admin] RBAC: Authorizing `identity:get_project(project_id=e574e273dabe4a3eab67dc95d42fcb6b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:05.066864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7d39a-2d48-4577-9f8b-1967b627ab81 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:05.068879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 184/375] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:04 2022] GET /identity/v3/projects/e574e273dabe4a3eab67dc95d42fcb6b => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:05.075516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7d39a-2d48-4577-9f8b-1967b627ab81 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:05.097678 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c686e89-026b-4c52-b189-3a88adb65ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:05.098327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686e89-026b-4c52-b189-3a88adb65ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:05.098606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686e89-026b-4c52-b189-3a88adb65ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:05.098876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c686e89-026b-4c52-b189-3a88adb65ef8 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:05.100460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c686e89-026b-4c52-b189-3a88adb65ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:05.102229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c686e89-026b-4c52-b189-3a88adb65ef8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:05.103918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 192/376] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:04 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:05.110299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c686e89-026b-4c52-b189-3a88adb65ef8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:05.195658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:05.270134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7d39a-2d48-4577-9f8b-1967b627ab81 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:05.284966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:05.314229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c686e89-026b-4c52-b189-3a88adb65ef8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:05.346551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850da7c0-a62a-4a41-b7e9-3da153a9ebbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:05.347186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850da7c0-a62a-4a41-b7e9-3da153a9ebbc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:05.347462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850da7c0-a62a-4a41-b7e9-3da153a9ebbc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:05.347736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850da7c0-a62a-4a41-b7e9-3da153a9ebbc None admin] PATH_INFO: `/v3/groups/1114abe10cdd45dfbfd6f5e2b49e9e28` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:05.358699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850da7c0-a62a-4a41-b7e9-3da153a9ebbc None admin] RBAC: Authorizing `identity:get_group(group_id=1114abe10cdd45dfbfd6f5e2b49e9e28)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:05.360148 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850da7c0-a62a-4a41-b7e9-3da153a9ebbc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:05.361983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 185/377] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:05 2022] GET /identity/v3/groups/1114abe10cdd45dfbfd6f5e2b49e9e28 => generated 229 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:05.368884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850da7c0-a62a-4a41-b7e9-3da153a9ebbc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:05.374708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ca518-1b18-43dd-8590-af10cc245aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:05.375359 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca518-1b18-43dd-8590-af10cc245aae None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:05.375643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca518-1b18-43dd-8590-af10cc245aae None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:05.375911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca518-1b18-43dd-8590-af10cc245aae None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:05.384236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ca518-1b18-43dd-8590-af10cc245aae None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:05.385754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ca518-1b18-43dd-8590-af10cc245aae None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:05.387580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 193/378] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:05 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:05.400907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ca518-1b18-43dd-8590-af10cc245aae None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:05.523089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850da7c0-a62a-4a41-b7e9-3da153a9ebbc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:05.532748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ca518-1b18-43dd-8590-af10cc245aae None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:05.550146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:05.559185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da434cf-c136-40d6-adc8-1f6b25832f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:05.559805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da434cf-c136-40d6-adc8-1f6b25832f34 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:05.560083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da434cf-c136-40d6-adc8-1f6b25832f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:05.560353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da434cf-c136-40d6-adc8-1f6b25832f34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:05.562045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da434cf-c136-40d6-adc8-1f6b25832f34 None admin] RBAC: Adding query filter params (group.id=1114abe10cdd45dfbfd6f5e2b49e9e28, role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=e574e273dabe4a3eab67dc95d42fcb6b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:05.562358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da434cf-c136-40d6-adc8-1f6b25832f34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:05.563874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da434cf-c136-40d6-adc8-1f6b25832f34 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:05.571238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 186/379] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 30 19:16:05 2022] GET /identity/v3/role_assignments?group.id=1114abe10cdd45dfbfd6f5e2b49e9e28&scope.project.id=e574e273dabe4a3eab67dc95d42fcb6b&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:05.572889 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc9442-c1c4-44b3-8a2d-f28a1b39a008 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:05.573178 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc9442-c1c4-44b3-8a2d-f28a1b39a008 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:05.573548 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc9442-c1c4-44b3-8a2d-f28a1b39a008 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:05.575166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 187/380] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:05.575288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183a0f7-766d-45c2-9c39-80499615860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:05.575913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183a0f7-766d-45c2-9c39-80499615860e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:05.576191 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183a0f7-766d-45c2-9c39-80499615860e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:05.576549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183a0f7-766d-45c2-9c39-80499615860e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:05.577289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183a0f7-766d-45c2-9c39-80499615860e None admin] RBAC: Adding query filter params (group.id=b0ade2d70c52410fba43d25bd173c0d4, role.id=9c17da6cc0e54b67af7cb7fb68638a7e, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:05.577593 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183a0f7-766d-45c2-9c39-80499615860e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:05.579057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183a0f7-766d-45c2-9c39-80499615860e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:05.581188 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3114209-f584-4b3c-8be6-5fb19d0217ed None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:05.581676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3114209-f584-4b3c-8be6-5fb19d0217ed None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:05.581853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3114209-f584-4b3c-8be6-5fb19d0217ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:05.588882 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 194/381] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 30 19:16:05 2022] GET /identity/v3/role_assignments?group.id=b0ade2d70c52410fba43d25bd173c0d4&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=9c17da6cc0e54b67af7cb7fb68638a7e => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:05.614645 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b3114209-f584-4b3c-8be6-5fb19d0217ed None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:05.615871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b3114209-f584-4b3c-8be6-5fb19d0217ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:05.712320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 188/382] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:06.162783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183a0f7-766d-45c2-9c39-80499615860e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:06.184736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:06.258352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34bd2-f842-4f82-8944-9b5a45345720 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:06.258743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34bd2-f842-4f82-8944-9b5a45345720 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:06.258971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34bd2-f842-4f82-8944-9b5a45345720 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:06.260547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 189/383] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:06.266253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad465484-9cc5-411f-9684-66a3168b3d33 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:06.266530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad465484-9cc5-411f-9684-66a3168b3d33 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:06.266798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad465484-9cc5-411f-9684-66a3168b3d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:06.286053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ad465484-9cc5-411f-9684-66a3168b3d33 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:06.287482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ad465484-9cc5-411f-9684-66a3168b3d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:06.300125 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183a0f7-766d-45c2-9c39-80499615860e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:06.363728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a531f6d-faa8-4c79-87d5-4f2a03e2cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:06.364394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a531f6d-faa8-4c79-87d5-4f2a03e2cef5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:06.364668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a531f6d-faa8-4c79-87d5-4f2a03e2cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:06.364931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a531f6d-faa8-4c79-87d5-4f2a03e2cef5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:06.365967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 190/384] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:06.404151 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a531f6d-faa8-4c79-87d5-4f2a03e2cef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-4a531f6d-faa8-4c79-87d5-4f2a03e2cef5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:06.409602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:06.410499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:06.411718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 195/385] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:06.419454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad465484-9cc5-411f-9684-66a3168b3d33 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:06.493114 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:06.555924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad465484-9cc5-411f-9684-66a3168b3d33 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:06.607104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f317719-9537-46f8-9fea-98b87e2321ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:06.607104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f317719-9537-46f8-9fea-98b87e2321ee None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:06.607104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f317719-9537-46f8-9fea-98b87e2321ee None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:06.607104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f317719-9537-46f8-9fea-98b87e2321ee None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:06.607943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f317719-9537-46f8-9fea-98b87e2321ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:06.607943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f317719-9537-46f8-9fea-98b87e2321ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:06.607943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f317719-9537-46f8-9fea-98b87e2321ee None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:06.620326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 191/386] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:16:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:06.622147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a531f6d-faa8-4c79-87d5-4f2a03e2cef5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:06.745843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:06.757897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f317719-9537-46f8-9fea-98b87e2321ee None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:06.768977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a531f6d-faa8-4c79-87d5-4f2a03e2cef5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:06.809539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bad39-1576-4e25-b06d-5b04edbcb6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:06.811356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bad39-1576-4e25-b06d-5b04edbcb6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:06.811356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bad39-1576-4e25-b06d-5b04edbcb6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:06.811356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bad39-1576-4e25-b06d-5b04edbcb6b2 None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:06.811876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22bad39-1576-4e25-b06d-5b04edbcb6b2 None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:06.815999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22bad39-1576-4e25-b06d-5b04edbcb6b2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:06.819982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 196/387] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:06 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:06.827272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bad39-1576-4e25-b06d-5b04edbcb6b2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:06.951246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bad39-1576-4e25-b06d-5b04edbcb6b2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:06.972100 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f317719-9537-46f8-9fea-98b87e2321ee None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:06.989245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a461c6-7f5e-4532-8fdd-b8bbc060bee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:06.989886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a461c6-7f5e-4532-8fdd-b8bbc060bee7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:06.990158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a461c6-7f5e-4532-8fdd-b8bbc060bee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:06.990435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a461c6-7f5e-4532-8fdd-b8bbc060bee7 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:06.991575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a461c6-7f5e-4532-8fdd-b8bbc060bee7 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:06.993591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a461c6-7f5e-4532-8fdd-b8bbc060bee7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:06.995487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 197/388] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:06 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:07.002177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a461c6-7f5e-4532-8fdd-b8bbc060bee7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:07.006108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:07.034165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:07.047004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee130a-36f7-41e3-98b8-dfa15c7f06da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:07.047645 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee130a-36f7-41e3-98b8-dfa15c7f06da None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:07.047919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee130a-36f7-41e3-98b8-dfa15c7f06da None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:07.048186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee130a-36f7-41e3-98b8-dfa15c7f06da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:07.072239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee130a-36f7-41e3-98b8-dfa15c7f06da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c4ee130a-36f7-41e3-98b8-dfa15c7f06da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:07.083438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:07.084402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:07.085238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 192/389] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:16:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:07.091172 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee130a-36f7-41e3-98b8-dfa15c7f06da None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:07.154842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a461c6-7f5e-4532-8fdd-b8bbc060bee7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:07.203903 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edddfbd-ff8a-44d5-b0f3-97620867e2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:07.204685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edddfbd-ff8a-44d5-b0f3-97620867e2d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:07.205049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edddfbd-ff8a-44d5-b0f3-97620867e2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:07.205329 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edddfbd-ff8a-44d5-b0f3-97620867e2d2 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b/groups/b0ade2d70c52410fba43d25bd173c0d4/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:07.209415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edddfbd-ff8a-44d5-b0f3-97620867e2d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=63a660324ea34aa886a24836744d158b, group_id=b0ade2d70c52410fba43d25bd173c0d4, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:07.210853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edddfbd-ff8a-44d5-b0f3-97620867e2d2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:07.232478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 198/390] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 30 19:16:07 2022] PUT /identity/v3/projects/63a660324ea34aa886a24836744d158b/groups/b0ade2d70c52410fba43d25bd173c0d4/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:07.270927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee130a-36f7-41e3-98b8-dfa15c7f06da None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:07.311102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d6d2c-d113-4ffe-9288-b59aa44f09c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:07.312068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d6d2c-d113-4ffe-9288-b59aa44f09c8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:07.312551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d6d2c-d113-4ffe-9288-b59aa44f09c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:07.313000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d6d2c-d113-4ffe-9288-b59aa44f09c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:07.313840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d6d2c-d113-4ffe-9288-b59aa44f09c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:07.314317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d6d2c-d113-4ffe-9288-b59aa44f09c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:07.315983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d6d2c-d113-4ffe-9288-b59aa44f09c8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:07.327153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 193/391] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:07.332825 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edddfbd-ff8a-44d5-b0f3-97620867e2d2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:07.470554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:07.507306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edddfbd-ff8a-44d5-b0f3-97620867e2d2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:07.546799 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5c8ed-b375-429a-94bf-858a95423a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:07.547360 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5c8ed-b375-429a-94bf-858a95423a25 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:07.547635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5c8ed-b375-429a-94bf-858a95423a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:07.547901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5c8ed-b375-429a-94bf-858a95423a25 None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:07.549005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5c8ed-b375-429a-94bf-858a95423a25 None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:07.550434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5c8ed-b375-429a-94bf-858a95423a25 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:07.551980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 199/392] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:07 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:07.558451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d6d2c-d113-4ffe-9288-b59aa44f09c8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:07.596274 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839794c-1b98-4696-8164-12d13cdf81fb None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:07.596636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839794c-1b98-4696-8164-12d13cdf81fb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:07.596836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839794c-1b98-4696-8164-12d13cdf81fb None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:07.599050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 200/393] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:07.604110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a409c-35d7-4ba8-84bb-fab6c14091f6 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:07.604410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a409c-35d7-4ba8-84bb-fab6c14091f6 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:07.604693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a409c-35d7-4ba8-84bb-fab6c14091f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:07.607067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:07.631517 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-496a409c-35d7-4ba8-84bb-fab6c14091f6 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:07.632738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-496a409c-35d7-4ba8-84bb-fab6c14091f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:07.701560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d6d2c-d113-4ffe-9288-b59aa44f09c8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:07.762490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 201/394] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:07.766755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a27769-efc4-484e-b5eb-3a6c177b3ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:07.767503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a27769-efc4-484e-b5eb-3a6c177b3ffc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:07.767880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a27769-efc4-484e-b5eb-3a6c177b3ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:07.768248 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a27769-efc4-484e-b5eb-3a6c177b3ffc None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:07.769512 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a27769-efc4-484e-b5eb-3a6c177b3ffc None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:07.771095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a27769-efc4-484e-b5eb-3a6c177b3ffc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:07.776455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 194/395] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:07 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:07.779672 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a409c-35d7-4ba8-84bb-fab6c14091f6 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:07.866513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:07.900871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a409c-35d7-4ba8-84bb-fab6c14091f6 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:07.945786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3362b5-97e8-4887-bb38-8026fbc055aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:07.947026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3362b5-97e8-4887-bb38-8026fbc055aa None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:07.947307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3362b5-97e8-4887-bb38-8026fbc055aa None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:07.947633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3362b5-97e8-4887-bb38-8026fbc055aa None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b/groups/b0ade2d70c52410fba43d25bd173c0d4/roles/9c17da6cc0e54b67af7cb7fb68638a7e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:07.951390 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3362b5-97e8-4887-bb38-8026fbc055aa None admin] RBAC: Authorizing `identity:create_grant(project_id=63a660324ea34aa886a24836744d158b, group_id=b0ade2d70c52410fba43d25bd173c0d4, role_id=9c17da6cc0e54b67af7cb7fb68638a7e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:07.952852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3362b5-97e8-4887-bb38-8026fbc055aa None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:07.970412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 202/396] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 30 19:16:07 2022] PUT /identity/v3/projects/63a660324ea34aa886a24836744d158b/groups/b0ade2d70c52410fba43d25bd173c0d4/roles/9c17da6cc0e54b67af7cb7fb68638a7e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:08.152403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a27769-efc4-484e-b5eb-3a6c177b3ffc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:08.298177 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a27769-efc4-484e-b5eb-3a6c177b3ffc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:08.300771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:08.340189 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfad1a-ae47-426c-8e96-1cd12e6e3b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:08.341015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfad1a-ae47-426c-8e96-1cd12e6e3b48 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:08.341291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfad1a-ae47-426c-8e96-1cd12e6e3b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:08.341556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfad1a-ae47-426c-8e96-1cd12e6e3b48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:08.344724 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730ce56-1d6f-4ccb-ab4c-1ed4a3de5015 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:08.345183 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730ce56-1d6f-4ccb-ab4c-1ed4a3de5015 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:08.345261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730ce56-1d6f-4ccb-ab4c-1ed4a3de5015 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:08.346614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 203/397] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:16:08.356318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfad1a-ae47-426c-8e96-1cd12e6e3b48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7adfad1a-ae47-426c-8e96-1cd12e6e3b48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:08.361667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:08.362822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:08.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 195/398] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:08.364565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc18d5-8964-4621-a8f4-3bdd09fc4ae7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:08.364756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc18d5-8964-4621-a8f4-3bdd09fc4ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:08.365032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc18d5-8964-4621-a8f4-3bdd09fc4ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:08.369949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfad1a-ae47-426c-8e96-1cd12e6e3b48 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:08.391065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-55cc18d5-8964-4621-a8f4-3bdd09fc4ae7 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:08.392849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-55cc18d5-8964-4621-a8f4-3bdd09fc4ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:08.491486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfad1a-ae47-426c-8e96-1cd12e6e3b48 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:08.530218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4c360-2b08-4b34-8081-1ec094e11d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:08.531223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4c360-2b08-4b34-8081-1ec094e11d80 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:08.531761 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4c360-2b08-4b34-8081-1ec094e11d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:08.532033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4c360-2b08-4b34-8081-1ec094e11d80 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:08.533019 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4c360-2b08-4b34-8081-1ec094e11d80 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:08.533320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4c360-2b08-4b34-8081-1ec094e11d80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:08.534761 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4c360-2b08-4b34-8081-1ec094e11d80 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:08.544923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 196/399] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:16:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:08.545893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 204/400] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:08.557020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4c360-2b08-4b34-8081-1ec094e11d80 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:08.592131 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:08.691393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4c360-2b08-4b34-8081-1ec094e11d80 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:08.731781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e68216-6fd6-4604-b36a-3bae556e60a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:08.732538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e68216-6fd6-4604-b36a-3bae556e60a5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:08.732703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e68216-6fd6-4604-b36a-3bae556e60a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:08.732978 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e68216-6fd6-4604-b36a-3bae556e60a5 None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:08.734088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e68216-6fd6-4604-b36a-3bae556e60a5 None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:08.735574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e68216-6fd6-4604-b36a-3bae556e60a5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:08.737541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 197/401] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:08 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:08.743350 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc18d5-8964-4621-a8f4-3bdd09fc4ae7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:08.767637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7496ec-9ebb-48b5-9cc0-9c642ffee138 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:08.767957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7496ec-9ebb-48b5-9cc0-9c642ffee138 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:08.768177 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7496ec-9ebb-48b5-9cc0-9c642ffee138 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:08.769997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 198/402] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:08.776944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560444d-a5ab-418e-9d15-c7086790f5b3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:08.777287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560444d-a5ab-418e-9d15-c7086790f5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:08.777565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560444d-a5ab-418e-9d15-c7086790f5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:08.803414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3560444d-a5ab-418e-9d15-c7086790f5b3 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:08.804624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3560444d-a5ab-418e-9d15-c7086790f5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:08.833280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:08.919317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc18d5-8964-4621-a8f4-3bdd09fc4ae7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:08.959519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca2178-e61e-45ff-8894-ca7d4152b776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:08.960158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca2178-e61e-45ff-8894-ca7d4152b776 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:08.960463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca2178-e61e-45ff-8894-ca7d4152b776 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:08.960741 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca2178-e61e-45ff-8894-ca7d4152b776 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:08.961951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca2178-e61e-45ff-8894-ca7d4152b776 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:08.962371 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 199/403] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:08.968445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca2178-e61e-45ff-8894-ca7d4152b776 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:08.968445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 205/404] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:08 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:08.982614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560444d-a5ab-418e-9d15-c7086790f5b3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:09.022181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca2178-e61e-45ff-8894-ca7d4152b776 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:09.027836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:09.183724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560444d-a5ab-418e-9d15-c7086790f5b3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:09.194982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca2178-e61e-45ff-8894-ca7d4152b776 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:09.229475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:09.232942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27761b30-293c-4908-af5f-678627445d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:09.233580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27761b30-293c-4908-af5f-678627445d66 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:09.233864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27761b30-293c-4908-af5f-678627445d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:09.234133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27761b30-293c-4908-af5f-678627445d66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:09.245365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27761b30-293c-4908-af5f-678627445d66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-27761b30-293c-4908-af5f-678627445d66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:09.250273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:09.251231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:09.252062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 206/405] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:09.258722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27761b30-293c-4908-af5f-678627445d66 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:09.263920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcbf9c-574d-49c1-a8c7-d868d2224427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:09.268906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcbf9c-574d-49c1-a8c7-d868d2224427 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:09.269393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcbf9c-574d-49c1-a8c7-d868d2224427 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:09.269866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcbf9c-574d-49c1-a8c7-d868d2224427 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/groups/b0ade2d70c52410fba43d25bd173c0d4/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:09.272085 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dcbf9c-574d-49c1-a8c7-d868d2224427 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2c3fdc09a94be08d88fa3ebc44057f, group_id=b0ade2d70c52410fba43d25bd173c0d4, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:09.273846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dcbf9c-574d-49c1-a8c7-d868d2224427 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:09.293994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 200/406] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 30 19:16:08 2022] PUT /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/groups/b0ade2d70c52410fba43d25bd173c0d4/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:09.295228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcbf9c-574d-49c1-a8c7-d868d2224427 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:09.437499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27761b30-293c-4908-af5f-678627445d66 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:09.485053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9513d8d-8ffb-4ebe-a8df-c24f96eeedbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:09.486032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9513d8d-8ffb-4ebe-a8df-c24f96eeedbe None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:09.486577 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9513d8d-8ffb-4ebe-a8df-c24f96eeedbe None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:09.487104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9513d8d-8ffb-4ebe-a8df-c24f96eeedbe None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:09.487821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9513d8d-8ffb-4ebe-a8df-c24f96eeedbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:09.488126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9513d8d-8ffb-4ebe-a8df-c24f96eeedbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:09.489550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9513d8d-8ffb-4ebe-a8df-c24f96eeedbe None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:09.497567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:09.500891 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 207/407] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:09.507413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9513d8d-8ffb-4ebe-a8df-c24f96eeedbe None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:09.518502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcbf9c-574d-49c1-a8c7-d868d2224427 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:09.557109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b623f76-7b7d-420a-953b-c6c5eb672b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:09.558072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b623f76-7b7d-420a-953b-c6c5eb672b16 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:09.558539 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b623f76-7b7d-420a-953b-c6c5eb672b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:09.559015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b623f76-7b7d-420a-953b-c6c5eb672b16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:09.571891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b623f76-7b7d-420a-953b-c6c5eb672b16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:09.575575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1b623f76-7b7d-420a-953b-c6c5eb672b16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:09.578720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:09.579923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:09.580777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:09.583269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 201/408] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:16:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:09.589341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b623f76-7b7d-420a-953b-c6c5eb672b16 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:09.642288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9513d8d-8ffb-4ebe-a8df-c24f96eeedbe None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:09.690062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22e62a-ff2f-47d2-95e3-4c9e2b911d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:09.690711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22e62a-ff2f-47d2-95e3-4c9e2b911d64 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:09.690989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22e62a-ff2f-47d2-95e3-4c9e2b911d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:09.691261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22e62a-ff2f-47d2-95e3-4c9e2b911d64 None admin] PATH_INFO: `/v3/groups/1114abe10cdd45dfbfd6f5e2b49e9e28` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:09.693510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22e62a-ff2f-47d2-95e3-4c9e2b911d64 None admin] RBAC: Authorizing `identity:get_group(group_id=1114abe10cdd45dfbfd6f5e2b49e9e28)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:09.694954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22e62a-ff2f-47d2-95e3-4c9e2b911d64 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:09.700671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 208/409] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:09 2022] GET /identity/v3/groups/1114abe10cdd45dfbfd6f5e2b49e9e28 => generated 229 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:09.706242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22e62a-ff2f-47d2-95e3-4c9e2b911d64 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:09.760553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b623f76-7b7d-420a-953b-c6c5eb672b16 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:09.813162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3bbf5-fdd2-42f1-b604-9b2808da53d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:09.813690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3bbf5-fdd2-42f1-b604-9b2808da53d2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:09.813967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3bbf5-fdd2-42f1-b604-9b2808da53d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:09.814235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3bbf5-fdd2-42f1-b604-9b2808da53d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:09.814905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3bbf5-fdd2-42f1-b604-9b2808da53d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:09.815202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3bbf5-fdd2-42f1-b604-9b2808da53d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:09.816603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3bbf5-fdd2-42f1-b604-9b2808da53d2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:09.839405 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 202/410] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:09.840345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3bbf5-fdd2-42f1-b604-9b2808da53d2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:09.854342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:09.886217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22e62a-ff2f-47d2-95e3-4c9e2b911d64 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:09.918584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:09.941177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320c3a6-1afd-46e7-977c-c89c5908c5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:09.942140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320c3a6-1afd-46e7-977c-c89c5908c5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:09.942648 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320c3a6-1afd-46e7-977c-c89c5908c5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:09.943110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320c3a6-1afd-46e7-977c-c89c5908c5f1 None admin] PATH_INFO: `/v3/projects/e574e273dabe4a3eab67dc95d42fcb6b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:09.949740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320c3a6-1afd-46e7-977c-c89c5908c5f1 None admin] RBAC: Authorizing `identity:get_project(project_id=e574e273dabe4a3eab67dc95d42fcb6b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:09.951368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320c3a6-1afd-46e7-977c-c89c5908c5f1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:09.953764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 209/411] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:09 2022] GET /identity/v3/projects/e574e273dabe4a3eab67dc95d42fcb6b => generated 344 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:09.964388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320c3a6-1afd-46e7-977c-c89c5908c5f1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:10.027561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3bbf5-fdd2-42f1-b604-9b2808da53d2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:10.098336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320c3a6-1afd-46e7-977c-c89c5908c5f1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:10.105740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ef567-15b2-4b9e-8b40-c2b154ba427a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:10.106645 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ef567-15b2-4b9e-8b40-c2b154ba427a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:10.107137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ef567-15b2-4b9e-8b40-c2b154ba427a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:10.107598 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ef567-15b2-4b9e-8b40-c2b154ba427a None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:10.113057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ef567-15b2-4b9e-8b40-c2b154ba427a None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:10.114744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ef567-15b2-4b9e-8b40-c2b154ba427a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:10.116846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 203/412] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:09 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:10.122565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ef567-15b2-4b9e-8b40-c2b154ba427a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:10.146541 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b00b60-1dff-4f81-ba56-ba5ea920d0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:10.147858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b00b60-1dff-4f81-ba56-ba5ea920d0b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:10.148355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b00b60-1dff-4f81-ba56-ba5ea920d0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:10.148889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b00b60-1dff-4f81-ba56-ba5ea920d0b4 None admin] PATH_INFO: `/v3/projects/e574e273dabe4a3eab67dc95d42fcb6b/groups/1114abe10cdd45dfbfd6f5e2b49e9e28/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:10.157187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b00b60-1dff-4f81-ba56-ba5ea920d0b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e574e273dabe4a3eab67dc95d42fcb6b, group_id=1114abe10cdd45dfbfd6f5e2b49e9e28, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:10.158913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b00b60-1dff-4f81-ba56-ba5ea920d0b4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:10.175115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 210/413] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 30 19:16:09 2022] PUT /identity/v3/projects/e574e273dabe4a3eab67dc95d42fcb6b/groups/1114abe10cdd45dfbfd6f5e2b49e9e28/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:10.176899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b8cee-6caa-44cd-82eb-ba8f037f0df7 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:10.177382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b8cee-6caa-44cd-82eb-ba8f037f0df7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:10.177819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b8cee-6caa-44cd-82eb-ba8f037f0df7 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:10.179417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 211/414] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:10.186079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c8670-8e3b-4620-bce1-b17a2f9ba228 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:10.186079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c8670-8e3b-4620-bce1-b17a2f9ba228 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:10.186353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c8670-8e3b-4620-bce1-b17a2f9ba228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:10.208194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-402c8670-8e3b-4620-bce1-b17a2f9ba228 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:10.209707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-402c8670-8e3b-4620-bce1-b17a2f9ba228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:10.223147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:10.259974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ef567-15b2-4b9e-8b40-c2b154ba427a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:10.284857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:10.306644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 212/415] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:10.320204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa8dd2-9870-4233-b076-ee71cf8d0bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:10.321540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa8dd2-9870-4233-b076-ee71cf8d0bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:10.321851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa8dd2-9870-4233-b076-ee71cf8d0bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:10.322124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa8dd2-9870-4233-b076-ee71cf8d0bb0 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:10.323564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa8dd2-9870-4233-b076-ee71cf8d0bb0 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:10.325252 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa8dd2-9870-4233-b076-ee71cf8d0bb0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:10.327545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 204/416] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:10 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:10.335289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c8670-8e3b-4620-bce1-b17a2f9ba228 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:10.480914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c8670-8e3b-4620-bce1-b17a2f9ba228 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:10.516653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3459dc-8801-49be-b93f-1867e137c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:10.517312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3459dc-8801-49be-b93f-1867e137c077 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:10.517583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3459dc-8801-49be-b93f-1867e137c077 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:10.517848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3459dc-8801-49be-b93f-1867e137c077 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/groups/b0ade2d70c52410fba43d25bd173c0d4/roles/9c17da6cc0e54b67af7cb7fb68638a7e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:10.520060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3459dc-8801-49be-b93f-1867e137c077 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e2c3fdc09a94be08d88fa3ebc44057f, group_id=b0ade2d70c52410fba43d25bd173c0d4, role_id=9c17da6cc0e54b67af7cb7fb68638a7e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:10.521583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3459dc-8801-49be-b93f-1867e137c077 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:10.536808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 213/417] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Sep 30 19:16:10 2022] PUT /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f/groups/b0ade2d70c52410fba43d25bd173c0d4/roles/9c17da6cc0e54b67af7cb7fb68638a7e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:10.690873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa8dd2-9870-4233-b076-ee71cf8d0bb0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:10.826210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa8dd2-9870-4233-b076-ee71cf8d0bb0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:10.865729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166a934-ed52-4f85-97f2-acbde728da08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:10.866714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a934-ed52-4f85-97f2-acbde728da08 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:10.867209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a934-ed52-4f85-97f2-acbde728da08 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:10.867675 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a934-ed52-4f85-97f2-acbde728da08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:10.881510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4e91f-a92e-41e3-b1de-249881b23086 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:10.881817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4e91f-a92e-41e3-b1de-249881b23086 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:10.882092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4e91f-a92e-41e3-b1de-249881b23086 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:10.883166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166a934-ed52-4f85-97f2-acbde728da08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:10.883698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 214/418] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b166a934-ed52-4f85-97f2-acbde728da08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:10.888401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:10.889330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:10.890525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:10.891519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaf6f3-838e-472c-aab0-3cbec6c23a7d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:10.891519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaf6f3-838e-472c-aab0-3cbec6c23a7d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:10.891519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaf6f3-838e-472c-aab0-3cbec6c23a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:10.891711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 205/419] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:10.895566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166a934-ed52-4f85-97f2-acbde728da08 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:10.906964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:10.920847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8afaf6f3-838e-472c-aab0-3cbec6c23a7d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:10.922699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8afaf6f3-838e-472c-aab0-3cbec6c23a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:10.961004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:11.032964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166a934-ed52-4f85-97f2-acbde728da08 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:11.062801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 215/420] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:11.086191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc54da-83ab-4ab2-a660-739dacd84279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:11.087212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc54da-83ab-4ab2-a660-739dacd84279 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:11.087685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc54da-83ab-4ab2-a660-739dacd84279 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:11.088149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc54da-83ab-4ab2-a660-739dacd84279 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:11.089037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc54da-83ab-4ab2-a660-739dacd84279 None admin] RBAC: Adding query filter params (name=member) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:11.089551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc54da-83ab-4ab2-a660-739dacd84279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:11.091111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc54da-83ab-4ab2-a660-739dacd84279 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:11.101597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 206/421] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:16:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:11.107453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaf6f3-838e-472c-aab0-3cbec6c23a7d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:11.266780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaf6f3-838e-472c-aab0-3cbec6c23a7d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:11.313044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7f14e-811e-41c9-aac7-2771281de7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:11.313654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7f14e-811e-41c9-aac7-2771281de7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:11.313931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7f14e-811e-41c9-aac7-2771281de7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:11.314198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7f14e-811e-41c9-aac7-2771281de7d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:11.361261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:11.396159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7f14e-811e-41c9-aac7-2771281de7d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:11.397828 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7f14e-811e-41c9-aac7-2771281de7d0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:11.465165 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc54da-83ab-4ab2-a660-739dacd84279 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:11.559446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 216/422] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:11.565625 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7f14e-811e-41c9-aac7-2771281de7d0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:11.592325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:11.666533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc54da-83ab-4ab2-a660-739dacd84279 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:11.696441 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:11.741454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a9fb4-b27b-4ed6-bca8-b1be7bd5e1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:11.742119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a9fb4-b27b-4ed6-bca8-b1be7bd5e1bb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:11.742402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a9fb4-b27b-4ed6-bca8-b1be7bd5e1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:11.742674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a9fb4-b27b-4ed6-bca8-b1be7bd5e1bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:11.758998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a9fb4-b27b-4ed6-bca8-b1be7bd5e1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:11.768245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7f14e-811e-41c9-aac7-2771281de7d0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5e7a9fb4-b27b-4ed6-bca8-b1be7bd5e1bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:11.772339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:11.773280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:11.774545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 207/423] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:16:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:11.795151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a9fb4-b27b-4ed6-bca8-b1be7bd5e1bb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:11.841258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec842b-8668-4747-a63c-8f124541da45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:11.841790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec842b-8668-4747-a63c-8f124541da45 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:11.842070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec842b-8668-4747-a63c-8f124541da45 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:11.842344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec842b-8668-4747-a63c-8f124541da45 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:11.844659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec842b-8668-4747-a63c-8f124541da45 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:11.847047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec842b-8668-4747-a63c-8f124541da45 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:11.848963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 217/424] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:11 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:11.850152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72b6de-d081-4332-9906-2813370eebe1 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:11.850433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72b6de-d081-4332-9906-2813370eebe1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:11.850703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72b6de-d081-4332-9906-2813370eebe1 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:11.851895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 218/425] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:11.857487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033823-6f1b-419f-94fb-7901fb7d7593 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:11.857775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033823-6f1b-419f-94fb-7901fb7d7593 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:11.858057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033823-6f1b-419f-94fb-7901fb7d7593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:11.876947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e0033823-6f1b-419f-94fb-7901fb7d7593 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:11.878140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e0033823-6f1b-419f-94fb-7901fb7d7593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:11.929800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a9fb4-b27b-4ed6-bca8-b1be7bd5e1bb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:11.956482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 219/426] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:11.957383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:11.957491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0033823-6f1b-419f-94fb-7901fb7d7593 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:11.971296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800d481-b024-415c-933e-71c3bf029a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:11.972222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800d481-b024-415c-933e-71c3bf029a4d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:11.972680 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800d481-b024-415c-933e-71c3bf029a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:11.973114 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800d481-b024-415c-933e-71c3bf029a4d None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:11.973963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800d481-b024-415c-933e-71c3bf029a4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:11.974457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800d481-b024-415c-933e-71c3bf029a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:11.975986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800d481-b024-415c-933e-71c3bf029a4d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:11.986047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 208/427] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:11.996601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800d481-b024-415c-933e-71c3bf029a4d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:12.074594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:12.134947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0033823-6f1b-419f-94fb-7901fb7d7593 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:12.192013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800d481-b024-415c-933e-71c3bf029a4d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:12.199250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd714c-4168-41d6-819e-f5202fbd79e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:12.200259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd714c-4168-41d6-819e-f5202fbd79e4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:12.201590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd714c-4168-41d6-819e-f5202fbd79e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:12.202038 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd714c-4168-41d6-819e-f5202fbd79e4 None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:12.205470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd714c-4168-41d6-819e-f5202fbd79e4 None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:12.207072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd714c-4168-41d6-819e-f5202fbd79e4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:12.210724 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 220/428] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:11 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:12.216503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd714c-4168-41d6-819e-f5202fbd79e4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:12.265651 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad32a40-2199-43e3-ba76-79d47d9215c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:12.266749 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad32a40-2199-43e3-ba76-79d47d9215c3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:12.268154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad32a40-2199-43e3-ba76-79d47d9215c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:12.269260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad32a40-2199-43e3-ba76-79d47d9215c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:12.339784 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:12.356059 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad32a40-2199-43e3-ba76-79d47d9215c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:12.356407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd714c-4168-41d6-819e-f5202fbd79e4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:12.357563 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad32a40-2199-43e3-ba76-79d47d9215c3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:12.416355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474c836-0abf-4074-9a4b-273a67b01420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:12.417588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474c836-0abf-4074-9a4b-273a67b01420 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:12.418075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474c836-0abf-4074-9a4b-273a67b01420 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:12.418781 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474c836-0abf-4074-9a4b-273a67b01420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:12.420079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474c836-0abf-4074-9a4b-273a67b01420 None admin] RBAC: Adding query filter params (group.id=b0ade2d70c52410fba43d25bd173c0d4, role.id=43452aaa2c9c4505b4bd3dd1b2412c65, scope.project.id=63a660324ea34aa886a24836744d158b) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:12.420607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474c836-0abf-4074-9a4b-273a67b01420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:12.422746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474c836-0abf-4074-9a4b-273a67b01420 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:12.425323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:12.433406 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 221/429] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 30 19:16:12 2022] GET /identity/v3/role_assignments?group.id=b0ade2d70c52410fba43d25bd173c0d4&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 611 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:12.434934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474c836-0abf-4074-9a4b-273a67b01420 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:12.480344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 209/430] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 30 19:16:12.485521 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad32a40-2199-43e3-ba76-79d47d9215c3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:12.567784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474c836-0abf-4074-9a4b-273a67b01420 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:12.613764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3980c-48e9-4588-9905-462f75b34e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:12.615036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3980c-48e9-4588-9905-462f75b34e04 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:12.615513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3980c-48e9-4588-9905-462f75b34e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:12.616014 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3980c-48e9-4588-9905-462f75b34e04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:12.620200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:12.632337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3980c-48e9-4588-9905-462f75b34e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-fcb3980c-48e9-4588-9905-462f75b34e04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:12.639111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:12.640090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 19:16:12.640952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:12.643875 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 222/431] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:12.653634 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3980c-48e9-4588-9905-462f75b34e04 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:12.663858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad32a40-2199-43e3-ba76-79d47d9215c3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:12.709587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a52c84-bb6c-4697-a95b-78dbbf7f41c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:12.710345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a52c84-bb6c-4697-a95b-78dbbf7f41c2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:12.710622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a52c84-bb6c-4697-a95b-78dbbf7f41c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:12.710889 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a52c84-bb6c-4697-a95b-78dbbf7f41c2 None admin] PATH_INFO: `/v3/projects/63a660324ea34aa886a24836744d158b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:12.713089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a52c84-bb6c-4697-a95b-78dbbf7f41c2 None admin] RBAC: Authorizing `identity:get_project(project_id=63a660324ea34aa886a24836744d158b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:12.714631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a52c84-bb6c-4697-a95b-78dbbf7f41c2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:12.716594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 210/432] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:12 2022] GET /identity/v3/projects/63a660324ea34aa886a24836744d158b => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:12.721488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a52c84-bb6c-4697-a95b-78dbbf7f41c2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:12.734035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:12.786455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3980c-48e9-4588-9905-462f75b34e04 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:12.843252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f60a1-6f6f-4672-be72-29cb01725c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:12.844240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f60a1-6f6f-4672-be72-29cb01725c0c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:12.844791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f60a1-6f6f-4672-be72-29cb01725c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:12.845280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f60a1-6f6f-4672-be72-29cb01725c0c None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:12.846141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f60a1-6f6f-4672-be72-29cb01725c0c None admin] RBAC: Adding query filter params (name=member) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:12.846628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f60a1-6f6f-4672-be72-29cb01725c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:12.848207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f60a1-6f6f-4672-be72-29cb01725c0c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:12.859108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 223/433] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:16:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:12.865226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f60a1-6f6f-4672-be72-29cb01725c0c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:12.865824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a52c84-bb6c-4697-a95b-78dbbf7f41c2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:12.907109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74e20b-94a3-4790-aab3-b15d546590c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:12.907689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74e20b-94a3-4790-aab3-b15d546590c7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:12.907971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74e20b-94a3-4790-aab3-b15d546590c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:12.908244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74e20b-94a3-4790-aab3-b15d546590c7 None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:12.909471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74e20b-94a3-4790-aab3-b15d546590c7 None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:12.911616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74e20b-94a3-4790-aab3-b15d546590c7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:12.913608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 211/434] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:12 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:12.924396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74e20b-94a3-4790-aab3-b15d546590c7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:12.962155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:13.002409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f60a1-6f6f-4672-be72-29cb01725c0c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:13.028126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:13.047356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ef5be-30ae-4bf2-b4e9-691a6c8a7498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:13.048305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ef5be-30ae-4bf2-b4e9-691a6c8a7498 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.048856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ef5be-30ae-4bf2-b4e9-691a6c8a7498 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.049269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ef5be-30ae-4bf2-b4e9-691a6c8a7498 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.082344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74e20b-94a3-4790-aab3-b15d546590c7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:13.119702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d60ca-1d9c-4c62-b75c-471fe832a564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:13.120301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d60ca-1d9c-4c62-b75c-471fe832a564 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.120604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d60ca-1d9c-4c62-b75c-471fe832a564 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.120873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d60ca-1d9c-4c62-b75c-471fe832a564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.121601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d60ca-1d9c-4c62-b75c-471fe832a564 None admin] RBAC: Adding query filter params (group.id=b0ade2d70c52410fba43d25bd173c0d4, role.id=9c17da6cc0e54b67af7cb7fb68638a7e, scope.project.id=63a660324ea34aa886a24836744d158b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:13.121896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d60ca-1d9c-4c62-b75c-471fe832a564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:13.123327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d60ca-1d9c-4c62-b75c-471fe832a564 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:13.128235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ef5be-30ae-4bf2-b4e9-691a6c8a7498 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:13.129711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ef5be-30ae-4bf2-b4e9-691a6c8a7498 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:13.130703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 212/435] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 30 19:16:12 2022] GET /identity/v3/role_assignments?group.id=b0ade2d70c52410fba43d25bd173c0d4&scope.project.id=63a660324ea34aa886a24836744d158b&role.id=9c17da6cc0e54b67af7cb7fb68638a7e => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:13.131914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a065a6-4f97-4324-b1cb-9524f8af31ff None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.132955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a065a6-4f97-4324-b1cb-9524f8af31ff None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.132955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a065a6-4f97-4324-b1cb-9524f8af31ff None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.134187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 213/436] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:13.135194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234061f4-b75c-418f-b002-33b74b588551 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.135463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234061f4-b75c-418f-b002-33b74b588551 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.135729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234061f4-b75c-418f-b002-33b74b588551 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.137243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 214/437] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:13.139775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfff2c8-381c-4090-9305-75a5409a6dd6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.140164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfff2c8-381c-4090-9305-75a5409a6dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.140560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfff2c8-381c-4090-9305-75a5409a6dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.160816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8bfff2c8-381c-4090-9305-75a5409a6dd6 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:13.161970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8bfff2c8-381c-4090-9305-75a5409a6dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:13.248396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 215/438] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:13.249960 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987a742-bd8f-40a3-8db9-e9a9dd9af4d6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.250243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987a742-bd8f-40a3-8db9-e9a9dd9af4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.250511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987a742-bd8f-40a3-8db9-e9a9dd9af4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.260808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 224/439] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:13.266169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ef5be-30ae-4bf2-b4e9-691a6c8a7498 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:13.270343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a987a742-bd8f-40a3-8db9-e9a9dd9af4d6 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:13.271366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a987a742-bd8f-40a3-8db9-e9a9dd9af4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:13.284159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:13.315534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:13.345461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 216/440] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:13.386604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ef5be-30ae-4bf2-b4e9-691a6c8a7498 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:13.430133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563a9bc-5cf0-4b84-87a3-5db9f3ee4123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:13.431411 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563a9bc-5cf0-4b84-87a3-5db9f3ee4123 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.431411 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563a9bc-5cf0-4b84-87a3-5db9f3ee4123 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.431411 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563a9bc-5cf0-4b84-87a3-5db9f3ee4123 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.434524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563a9bc-5cf0-4b84-87a3-5db9f3ee4123 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:13.434524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563a9bc-5cf0-4b84-87a3-5db9f3ee4123 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:13.435673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 225/441] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:13 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:13.441458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987a742-bd8f-40a3-8db9-e9a9dd9af4d6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:13.527745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563a9bc-5cf0-4b84-87a3-5db9f3ee4123 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:13.549477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987a742-bd8f-40a3-8db9-e9a9dd9af4d6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:13.592924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34277c-fdc7-4d69-b56d-b665db05fa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:13.593556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34277c-fdc7-4d69-b56d-b665db05fa5e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.593830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34277c-fdc7-4d69-b56d-b665db05fa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.594102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34277c-fdc7-4d69-b56d-b665db05fa5e None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.596306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34277c-fdc7-4d69-b56d-b665db05fa5e None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:13.597765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34277c-fdc7-4d69-b56d-b665db05fa5e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:13.599341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 217/442] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:13 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:13.606091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34277c-fdc7-4d69-b56d-b665db05fa5e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:13.624722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:13.651996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:13.666447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563a9bc-5cf0-4b84-87a3-5db9f3ee4123 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:13.700978 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cdb5b-a866-4ab0-bf73-81a207d4ceac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:13.701601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cdb5b-a866-4ab0-bf73-81a207d4ceac None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.701877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cdb5b-a866-4ab0-bf73-81a207d4ceac None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.702143 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cdb5b-a866-4ab0-bf73-81a207d4ceac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.712188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96cdb5b-a866-4ab0-bf73-81a207d4ceac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:13.716142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34277c-fdc7-4d69-b56d-b665db05fa5e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-b96cdb5b-a866-4ab0-bf73-81a207d4ceac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:13.716973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:13.717918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:13.718723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 226/443] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:16:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:13.719440 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cdb5b-a866-4ab0-bf73-81a207d4ceac None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:13.753935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb5060-3862-4ceb-90a0-1394b143848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:13.754546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb5060-3862-4ceb-90a0-1394b143848f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.754803 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb5060-3862-4ceb-90a0-1394b143848f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.755068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb5060-3862-4ceb-90a0-1394b143848f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.755775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb5060-3862-4ceb-90a0-1394b143848f None admin] RBAC: Adding query filter params (group.id=b0ade2d70c52410fba43d25bd173c0d4, role.id=43452aaa2c9c4505b4bd3dd1b2412c65, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:13.756075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb5060-3862-4ceb-90a0-1394b143848f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:13.757523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb5060-3862-4ceb-90a0-1394b143848f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:13.763446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 218/444] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 30 19:16:13 2022] GET /identity/v3/role_assignments?group.id=b0ade2d70c52410fba43d25bd173c0d4&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=43452aaa2c9c4505b4bd3dd1b2412c65 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:13.764090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb5060-3862-4ceb-90a0-1394b143848f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:13.835479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cdb5b-a866-4ab0-bf73-81a207d4ceac None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:13.855033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:13.873154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6844e-ea31-4acc-b2b0-370ff9f964fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:13.873803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6844e-ea31-4acc-b2b0-370ff9f964fc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.874079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6844e-ea31-4acc-b2b0-370ff9f964fc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.874564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6844e-ea31-4acc-b2b0-370ff9f964fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.880591 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb5060-3862-4ceb-90a0-1394b143848f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:13.886681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce6844e-ea31-4acc-b2b0-370ff9f964fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-cce6844e-ea31-4acc-b2b0-370ff9f964fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:13.891633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:13.892681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:13.893526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 227/445] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:13.898848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6844e-ea31-4acc-b2b0-370ff9f964fc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:13.911933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:13.920842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71981bd8-0337-40c9-b323-99a2e653d3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:13.921454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71981bd8-0337-40c9-b323-99a2e653d3af None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:13.921733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71981bd8-0337-40c9-b323-99a2e653d3af None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:13.921999 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71981bd8-0337-40c9-b323-99a2e653d3af None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:13.922670 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71981bd8-0337-40c9-b323-99a2e653d3af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:13.922968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71981bd8-0337-40c9-b323-99a2e653d3af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:13.924399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71981bd8-0337-40c9-b323-99a2e653d3af None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:13.933458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 219/446] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:13.940082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71981bd8-0337-40c9-b323-99a2e653d3af None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:14.018969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6844e-ea31-4acc-b2b0-370ff9f964fc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:14.059149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1ced6-8199-4ecf-8d22-f75d82b6e537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:14.059751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1ced6-8199-4ecf-8d22-f75d82b6e537 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:14.060021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1ced6-8199-4ecf-8d22-f75d82b6e537 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:14.060287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1ced6-8199-4ecf-8d22-f75d82b6e537 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:14.061020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1ced6-8199-4ecf-8d22-f75d82b6e537 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:14.061316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1ced6-8199-4ecf-8d22-f75d82b6e537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:14.065413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1ced6-8199-4ecf-8d22-f75d82b6e537 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:14.065541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71981bd8-0337-40c9-b323-99a2e653d3af None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:14.072207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 228/447] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:14.077509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1ced6-8199-4ecf-8d22-f75d82b6e537 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:14.105484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718e450-516a-406e-9fe5-60a1394c8ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:14.106099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718e450-516a-406e-9fe5-60a1394c8ddc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:14.106374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718e450-516a-406e-9fe5-60a1394c8ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:14.106649 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718e450-516a-406e-9fe5-60a1394c8ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:14.119556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:14.159303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:14.181722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718e450-516a-406e-9fe5-60a1394c8ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:14.183182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718e450-516a-406e-9fe5-60a1394c8ddc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:14.191272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1ced6-8199-4ecf-8d22-f75d82b6e537 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:14.222976 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcdd23-2069-4320-a046-3034f8ca6b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:14.223577 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcdd23-2069-4320-a046-3034f8ca6b9d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:14.223856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcdd23-2069-4320-a046-3034f8ca6b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:14.224156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcdd23-2069-4320-a046-3034f8ca6b9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:14.277273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 220/448] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:14.282797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718e450-516a-406e-9fe5-60a1394c8ddc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:14.289671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcdd23-2069-4320-a046-3034f8ca6b9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:14.291154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcdd23-2069-4320-a046-3034f8ca6b9d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:14.323547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:14.356754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:14.380676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718e450-516a-406e-9fe5-60a1394c8ddc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:14.391278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 229/449] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 30 19:16:14.399756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcdd23-2069-4320-a046-3034f8ca6b9d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:14.414385 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb13d0-dd25-4f7f-8f6d-ef608b864711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:14.415072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb13d0-dd25-4f7f-8f6d-ef608b864711 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:14.415347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb13d0-dd25-4f7f-8f6d-ef608b864711 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:14.415611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb13d0-dd25-4f7f-8f6d-ef608b864711 None admin] PATH_INFO: `/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:14.416742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb13d0-dd25-4f7f-8f6d-ef608b864711 None admin] RBAC: Authorizing `identity:get_project(project_id=1e2c3fdc09a94be08d88fa3ebc44057f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:14.418159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb13d0-dd25-4f7f-8f6d-ef608b864711 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:14.419760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 221/450] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:14 2022] GET /identity/v3/projects/1e2c3fdc09a94be08d88fa3ebc44057f => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:14.425055 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb13d0-dd25-4f7f-8f6d-ef608b864711 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:14.492612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcdd23-2069-4320-a046-3034f8ca6b9d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:14.520547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb13d0-dd25-4f7f-8f6d-ef608b864711 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:14.526005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a30388-164a-4df5-bfc9-5406a9a9eccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:14.526633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30388-164a-4df5-bfc9-5406a9a9eccc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:14.526912 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30388-164a-4df5-bfc9-5406a9a9eccc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:14.527184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30388-164a-4df5-bfc9-5406a9a9eccc None admin] PATH_INFO: `/v3/projects/e574e273dabe4a3eab67dc95d42fcb6b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:14.528358 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a30388-164a-4df5-bfc9-5406a9a9eccc None admin] RBAC: Authorizing `identity:get_project(project_id=e574e273dabe4a3eab67dc95d42fcb6b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:14.529407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:14.529760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a30388-164a-4df5-bfc9-5406a9a9eccc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:14.531873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 230/451] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:16:14 2022] GET /identity/v3/projects/e574e273dabe4a3eab67dc95d42fcb6b => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:14.536867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a30388-164a-4df5-bfc9-5406a9a9eccc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:14.554025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b7d6f-8dc0-4e2f-ade7-4d4f25016012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:14.554664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b7d6f-8dc0-4e2f-ade7-4d4f25016012 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:14.554952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b7d6f-8dc0-4e2f-ade7-4d4f25016012 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:14.555716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b7d6f-8dc0-4e2f-ade7-4d4f25016012 None admin] PATH_INFO: `/v3/groups/b0ade2d70c52410fba43d25bd173c0d4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:14.556866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b7d6f-8dc0-4e2f-ade7-4d4f25016012 None admin] RBAC: Authorizing `identity:get_group(group_id=b0ade2d70c52410fba43d25bd173c0d4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:14.558284 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b7d6f-8dc0-4e2f-ade7-4d4f25016012 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:14.559830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 222/452] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:14 2022] GET /identity/v3/groups/b0ade2d70c52410fba43d25bd173c0d4 => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:14.565504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b7d6f-8dc0-4e2f-ade7-4d4f25016012 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:14.577259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:14.635917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a30388-164a-4df5-bfc9-5406a9a9eccc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:14.663155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b7d6f-8dc0-4e2f-ade7-4d4f25016012 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:14.669510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccff17-5764-4f10-bb9f-da1c06728f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:14.670135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccff17-5764-4f10-bb9f-da1c06728f20 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:14.670408 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccff17-5764-4f10-bb9f-da1c06728f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:14.670688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccff17-5764-4f10-bb9f-da1c06728f20 None admin] PATH_INFO: `/v3/groups/1114abe10cdd45dfbfd6f5e2b49e9e28` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:14.671780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccff17-5764-4f10-bb9f-da1c06728f20 None admin] RBAC: Authorizing `identity:get_group(group_id=1114abe10cdd45dfbfd6f5e2b49e9e28)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:14.673259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccff17-5764-4f10-bb9f-da1c06728f20 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:14.674950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 231/453] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Sep 30 19:16:14 2022] GET /identity/v3/groups/1114abe10cdd45dfbfd6f5e2b49e9e28 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:14.680693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccff17-5764-4f10-bb9f-da1c06728f20 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:14.697400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61f3bd-e1b7-4df1-9701-9c7dc86f8182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:14.697991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f3bd-e1b7-4df1-9701-9c7dc86f8182 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:14.698263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f3bd-e1b7-4df1-9701-9c7dc86f8182 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:14.698530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f3bd-e1b7-4df1-9701-9c7dc86f8182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:14.699243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61f3bd-e1b7-4df1-9701-9c7dc86f8182 None admin] RBAC: Adding query filter params (group.id=b0ade2d70c52410fba43d25bd173c0d4, role.id=9c17da6cc0e54b67af7cb7fb68638a7e, scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:14.699542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61f3bd-e1b7-4df1-9701-9c7dc86f8182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:14.700982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61f3bd-e1b7-4df1-9701-9c7dc86f8182 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:14.706791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 223/454] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 30 19:16:14 2022] GET /identity/v3/role_assignments?group.id=b0ade2d70c52410fba43d25bd173c0d4&scope.project.id=1e2c3fdc09a94be08d88fa3ebc44057f&role.id=9c17da6cc0e54b67af7cb7fb68638a7e => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:14.785923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccff17-5764-4f10-bb9f-da1c06728f20 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:14.794351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:14.822098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78f1ab-5317-40cd-8e53-9ce314e61cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:14.822715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78f1ab-5317-40cd-8e53-9ce314e61cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:14.822991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78f1ab-5317-40cd-8e53-9ce314e61cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:14.823261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78f1ab-5317-40cd-8e53-9ce314e61cc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:14.823990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78f1ab-5317-40cd-8e53-9ce314e61cc9 None admin] RBAC: Adding query filter params (group.id=1114abe10cdd45dfbfd6f5e2b49e9e28, role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=e574e273dabe4a3eab67dc95d42fcb6b) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:14.824293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78f1ab-5317-40cd-8e53-9ce314e61cc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:14.825801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78f1ab-5317-40cd-8e53-9ce314e61cc9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:14.832302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 232/455] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Sep 30 19:16:14 2022] GET /identity/v3/role_assignments?group.id=1114abe10cdd45dfbfd6f5e2b49e9e28&scope.project.id=e574e273dabe4a3eab67dc95d42fcb6b&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:17.375134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be664a01-6e0c-4453-8420-4380c1100b03 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:17.375647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be664a01-6e0c-4453-8420-4380c1100b03 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:17.375647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be664a01-6e0c-4453-8420-4380c1100b03 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:17.377475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 224/456] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:17.382414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a0f03-2772-4ba0-b4e2-9df7b77e27fc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:17.382815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a0f03-2772-4ba0-b4e2-9df7b77e27fc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:17.382954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a0f03-2772-4ba0-b4e2-9df7b77e27fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:17.409956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-456a0f03-2772-4ba0-b4e2-9df7b77e27fc None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:17.411146 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-456a0f03-2772-4ba0-b4e2-9df7b77e27fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:17.514980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 233/457] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:17.699983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619de23-e2a9-418d-b58a-7615c91f998b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:17.700681 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619de23-e2a9-418d-b58a-7615c91f998b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:17.701184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619de23-e2a9-418d-b58a-7615c91f998b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:17.702919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 225/458] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:17.708717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832ad31-d810-4e95-a103-b8c2fed1425d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:17.709251 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832ad31-d810-4e95-a103-b8c2fed1425d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:17.709757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832ad31-d810-4e95-a103-b8c2fed1425d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:17.733304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7832ad31-d810-4e95-a103-b8c2fed1425d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:17.734796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7832ad31-d810-4e95-a103-b8c2fed1425d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:17.829984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 234/459] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:17.886280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619de23-e2a9-418d-b58a-7615c91f998b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:17.938728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:17.992188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850152a8-2b84-4c3a-9f14-b7b125540efb None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:17.992789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850152a8-2b84-4c3a-9f14-b7b125540efb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:17.993216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850152a8-2b84-4c3a-9f14-b7b125540efb None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:17.996359 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 235/460] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:18.001267 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ac761-2fb5-4b74-a2e9-dcc0a252fbae None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:18.001905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ac761-2fb5-4b74-a2e9-dcc0a252fbae None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:18.002388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ac761-2fb5-4b74-a2e9-dcc0a252fbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:18.024995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619de23-e2a9-418d-b58a-7615c91f998b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:18.035588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b59ac761-2fb5-4b74-a2e9-dcc0a252fbae None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:18.035588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b59ac761-2fb5-4b74-a2e9-dcc0a252fbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:18.078446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:18.090453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b2483-8913-435c-95a6-a7ac1e883375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:18.092143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b2483-8913-435c-95a6-a7ac1e883375 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:18.093626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b2483-8913-435c-95a6-a7ac1e883375 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:18.094267 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b2483-8913-435c-95a6-a7ac1e883375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:18.113762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b2483-8913-435c-95a6-a7ac1e883375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:18.115906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b2483-8913-435c-95a6-a7ac1e883375 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bd1b2483-8913-435c-95a6-a7ac1e883375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:18.138995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:18.140197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:18.141288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:18.141288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:18.141288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:18.142872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 226/461] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:18.147059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 236/462] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:18.147207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72034020-bfcf-423f-8801-02df6ce08c02 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:18.147207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72034020-bfcf-423f-8801-02df6ce08c02 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:18.147207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72034020-bfcf-423f-8801-02df6ce08c02 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:18.147398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ac761-2fb5-4b74-a2e9-dcc0a252fbae None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:18.147468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 227/463] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:18.150143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72034020-bfcf-423f-8801-02df6ce08c02 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:18.256317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ac761-2fb5-4b74-a2e9-dcc0a252fbae None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:18.272762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:18.282653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72034020-bfcf-423f-8801-02df6ce08c02 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:18.296079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb811930-b17e-414b-9c6d-b3eceae83e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:18.296893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb811930-b17e-414b-9c6d-b3eceae83e61 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:18.297315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb811930-b17e-414b-9c6d-b3eceae83e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:18.297680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb811930-b17e-414b-9c6d-b3eceae83e61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:18.310397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb811930-b17e-414b-9c6d-b3eceae83e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:18.312214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb811930-b17e-414b-9c6d-b3eceae83e61 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:18.325729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51790304-4ccb-4965-a5f1-eb4d0b04b1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:18.326349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51790304-4ccb-4965-a5f1-eb4d0b04b1fa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:18.326626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51790304-4ccb-4965-a5f1-eb4d0b04b1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:18.326894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51790304-4ccb-4965-a5f1-eb4d0b04b1fa None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:18.327531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51790304-4ccb-4965-a5f1-eb4d0b04b1fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:18.327827 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51790304-4ccb-4965-a5f1-eb4d0b04b1fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:18.329316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51790304-4ccb-4965-a5f1-eb4d0b04b1fa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-fb811930-b17e-414b-9c6d-b3eceae83e61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:18.329832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:18.330708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:18.331581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:18.331581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:18.331581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:18.336116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 237/464] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:18.337930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b562a-2896-419e-ac9d-4850276e65da None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:18.338212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b562a-2896-419e-ac9d-4850276e65da None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:18.338585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b562a-2896-419e-ac9d-4850276e65da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:18.342081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 228/465] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:18.342891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51790304-4ccb-4965-a5f1-eb4d0b04b1fa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:18.361004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-af2b562a-2896-419e-ac9d-4850276e65da None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:18.362090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-af2b562a-2896-419e-ac9d-4850276e65da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:18.373802 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:18.441225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 238/466] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:18.442077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b562a-2896-419e-ac9d-4850276e65da None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:18.469563 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51790304-4ccb-4965-a5f1-eb4d0b04b1fa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:18.505919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de28ff-9eed-40bb-8fdb-de7aef13a188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:18.506773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de28ff-9eed-40bb-8fdb-de7aef13a188 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:18.507230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de28ff-9eed-40bb-8fdb-de7aef13a188 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:18.507689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de28ff-9eed-40bb-8fdb-de7aef13a188 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:18.509152 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de28ff-9eed-40bb-8fdb-de7aef13a188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:18.509708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de28ff-9eed-40bb-8fdb-de7aef13a188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:18.511233 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de28ff-9eed-40bb-8fdb-de7aef13a188 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:18.524274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 229/467] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:18.525305 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de28ff-9eed-40bb-8fdb-de7aef13a188 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:18.544010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:18.566313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b562a-2896-419e-ac9d-4850276e65da None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:18.606354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0db582-fbb3-4521-86b7-65731517bd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:18.607074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0db582-fbb3-4521-86b7-65731517bd66 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:18.607417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0db582-fbb3-4521-86b7-65731517bd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:18.607760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0db582-fbb3-4521-86b7-65731517bd66 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:18.608506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0db582-fbb3-4521-86b7-65731517bd66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:18.610005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0db582-fbb3-4521-86b7-65731517bd66 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:18.632765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:18.639859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 239/468] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:16:18 2022] POST /identity/v3/users => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:16:18.641211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0db582-fbb3-4521-86b7-65731517bd66 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:18.644059 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de28ff-9eed-40bb-8fdb-de7aef13a188 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:18.713451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a128ef-2fbf-4a91-98a4-c84ae7068c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:18.714138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a128ef-2fbf-4a91-98a4-c84ae7068c5b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:18.714413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a128ef-2fbf-4a91-98a4-c84ae7068c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:18.714677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a128ef-2fbf-4a91-98a4-c84ae7068c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:18.725882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a128ef-2fbf-4a91-98a4-c84ae7068c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:18.727933 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a128ef-2fbf-4a91-98a4-c84ae7068c5b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-10a128ef-2fbf-4a91-98a4-c84ae7068c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:18.744437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:18.745463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:18.746355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:18.746355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:18.746355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:18.750811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 230/469] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:18.751774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a128ef-2fbf-4a91-98a4-c84ae7068c5b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:18.804082 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0db582-fbb3-4521-86b7-65731517bd66 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:18.825723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:18.852950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad31ec-18d3-4753-9e1a-df7041cb9051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:18.853649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad31ec-18d3-4753-9e1a-df7041cb9051 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:18.853929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad31ec-18d3-4753-9e1a-df7041cb9051 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:18.854258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad31ec-18d3-4753-9e1a-df7041cb9051 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:18.854905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad31ec-18d3-4753-9e1a-df7041cb9051 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:18.856327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad31ec-18d3-4753-9e1a-df7041cb9051 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:18.876987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a128ef-2fbf-4a91-98a4-c84ae7068c5b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:18.881570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 240/470] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:16:18 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:16:18.882519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad31ec-18d3-4753-9e1a-df7041cb9051 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:18.936277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d903e-2477-40e5-82f1-1f5bce4a70e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:18.937020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d903e-2477-40e5-82f1-1f5bce4a70e9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:18.937383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d903e-2477-40e5-82f1-1f5bce4a70e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:18.937787 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d903e-2477-40e5-82f1-1f5bce4a70e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:18.949074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:18.950930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d903e-2477-40e5-82f1-1f5bce4a70e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:18.952545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d903e-2477-40e5-82f1-1f5bce4a70e9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d66d903e-2477-40e5-82f1-1f5bce4a70e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:18.969038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:18.970054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:18.970947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:18.970947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:18.970947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:18.975415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 231/471] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:18.976823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d903e-2477-40e5-82f1-1f5bce4a70e9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:19.004515 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad31ec-18d3-4753-9e1a-df7041cb9051 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:19.061281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206c801-a96c-4955-8ccd-e9871f587464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:19.061831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206c801-a96c-4955-8ccd-e9871f587464 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:19.062110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206c801-a96c-4955-8ccd-e9871f587464 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:19.062375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206c801-a96c-4955-8ccd-e9871f587464 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:19.063033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2206c801-a96c-4955-8ccd-e9871f587464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:19.063328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2206c801-a96c-4955-8ccd-e9871f587464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:19.064778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2206c801-a96c-4955-8ccd-e9871f587464 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:19.098591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 241/472] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:19.104762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206c801-a96c-4955-8ccd-e9871f587464 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:19.138793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d903e-2477-40e5-82f1-1f5bce4a70e9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:19.159531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:19.179995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97e1ea-8dd2-432b-8954-72955a851568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:19.180681 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97e1ea-8dd2-432b-8954-72955a851568 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:19.180953 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97e1ea-8dd2-432b-8954-72955a851568 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:19.181231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97e1ea-8dd2-432b-8954-72955a851568 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:19.181862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97e1ea-8dd2-432b-8954-72955a851568 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:19.182155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97e1ea-8dd2-432b-8954-72955a851568 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:19.183537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97e1ea-8dd2-432b-8954-72955a851568 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:19.200507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 232/473] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:19.202673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97e1ea-8dd2-432b-8954-72955a851568 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:19.243646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206c801-a96c-4955-8ccd-e9871f587464 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:19.262434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:19.290968 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea6cdc-a604-4a0f-87ad-86cc51e0a116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:19.292090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea6cdc-a604-4a0f-87ad-86cc51e0a116 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:19.292990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea6cdc-a604-4a0f-87ad-86cc51e0a116 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:19.293606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea6cdc-a604-4a0f-87ad-86cc51e0a116 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:19.294611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea6cdc-a604-4a0f-87ad-86cc51e0a116 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:19.296722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea6cdc-a604-4a0f-87ad-86cc51e0a116 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:19.321494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97e1ea-8dd2-432b-8954-72955a851568 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:19.325085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 242/474] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:16:19 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:16:19.342032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014de02-135e-4de3-ad30-9bf47b83a73d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:19.342351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014de02-135e-4de3-ad30-9bf47b83a73d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:19.342786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014de02-135e-4de3-ad30-9bf47b83a73d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:19.344756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 243/475] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:16:19.351893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4934bd9-4d8f-43eb-ab33-2a3289c75a29 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:19.352693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4934bd9-4d8f-43eb-ab33-2a3289c75a29 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:19.353116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4934bd9-4d8f-43eb-ab33-2a3289c75a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:19.369253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13ca63-0514-4d3a-8062-839104678931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:19.370696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13ca63-0514-4d3a-8062-839104678931 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:19.371337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13ca63-0514-4d3a-8062-839104678931 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:19.371884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13ca63-0514-4d3a-8062-839104678931 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:19.372763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13ca63-0514-4d3a-8062-839104678931 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:19.374578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13ca63-0514-4d3a-8062-839104678931 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:19.375427 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e4934bd9-4d8f-43eb-ab33-2a3289c75a29 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:19.376758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e4934bd9-4d8f-43eb-ab33-2a3289c75a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:19.420574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 233/476] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:16:19 2022] POST /identity/v3/users => generated 249 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:16:19.497862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 244/477] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:19.807317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13ca63-0514-4d3a-8062-839104678931 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:19.843119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:19.927006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13ca63-0514-4d3a-8062-839104678931 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:19.966523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3efdf8-3019-4bbc-951e-57b96270ae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:19.967129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efdf8-3019-4bbc-951e-57b96270ae92 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:19.967402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efdf8-3019-4bbc-951e-57b96270ae92 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:19.967670 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3efdf8-3019-4bbc-951e-57b96270ae92 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:19.968346 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3efdf8-3019-4bbc-951e-57b96270ae92 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:19.969793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3efdf8-3019-4bbc-951e-57b96270ae92 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:19.978287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.sql.core [None req-5b3efdf8-3019-4bbc-951e-57b96270ae92 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 30 19:16:19.978287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 30 19:16:19.978287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [parameters: {'id': 'b4aad1f38bf447f8b6201c1ac09d4f92', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 30 19:16:19.978287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81403) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5b3efdf8-3019-4bbc-951e-57b96270ae92 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 30 19:16:19.995475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return ref.to_dict() Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 19:16:19.996443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application yield resource Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.rollback() Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.force_reraise() Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise self.value Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application session.commit() Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 30 19:16:19.997460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._prepare_impl() Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.flush() Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._flush(objects) Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application compat.raise_( Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application flush_context.execute() Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rec.execute(self) Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 30 19:16:19.998357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 19:16:19.999287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 19:16:20.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [parameters: {'id': 'b4aad1f38bf447f8b6201c1ac09d4f92', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:20.002736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 30 19:16:20.004851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:20.006924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 234/478] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 30 19:16:19 2022] POST /identity/v3/roles => generated 144 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:20.007061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4934bd9-4d8f-43eb-ab33-2a3289c75a29 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:20.157824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:20.161133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4934bd9-4d8f-43eb-ab33-2a3289c75a29 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:20.197260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1911c-c3d1-40b6-8fd2-b760f292cedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:20.197893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1911c-c3d1-40b6-8fd2-b760f292cedf None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:20.198166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1911c-c3d1-40b6-8fd2-b760f292cedf None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:20.198432 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1911c-c3d1-40b6-8fd2-b760f292cedf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:20.208820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1911c-c3d1-40b6-8fd2-b760f292cedf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-85b1911c-c3d1-40b6-8fd2-b760f292cedf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:20.214271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:20.216342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:20.218287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 245/479] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:16:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:20.224235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3efdf8-3019-4bbc-951e-57b96270ae92 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:20.344598 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:20.351234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3efdf8-3019-4bbc-951e-57b96270ae92 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:20.390693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288c693-e42c-4a3e-87fb-4c69960cd139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:20.391312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288c693-e42c-4a3e-87fb-4c69960cd139 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:20.391587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288c693-e42c-4a3e-87fb-4c69960cd139 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:20.391854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288c693-e42c-4a3e-87fb-4c69960cd139 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:20.392565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288c693-e42c-4a3e-87fb-4c69960cd139 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:20.393153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288c693-e42c-4a3e-87fb-4c69960cd139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:20.394873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288c693-e42c-4a3e-87fb-4c69960cd139 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:20.405175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 235/480] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:21.395858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558d623-152d-4c40-8ad3-5c3153e9d312 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:21.401147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558d623-152d-4c40-8ad3-5c3153e9d312 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:21.401147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558d623-152d-4c40-8ad3-5c3153e9d312 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:21.401147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 246/481] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:21.407351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c34f8-8e70-4178-a300-2e222bb59d96 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:21.407351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c34f8-8e70-4178-a300-2e222bb59d96 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:21.407351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c34f8-8e70-4178-a300-2e222bb59d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:21.433351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-134c34f8-8e70-4178-a300-2e222bb59d96 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:21.434538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-134c34f8-8e70-4178-a300-2e222bb59d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:21.530935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 236/482] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:21.743443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4247b3-cd3c-47dd-b99d-803bed2ce32f None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:21.743987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4247b3-cd3c-47dd-b99d-803bed2ce32f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:21.743987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4247b3-cd3c-47dd-b99d-803bed2ce32f None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:21.746217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 247/483] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:21.762010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84042748-c0f6-4dbf-8779-74fa0379ca1c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:21.762290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84042748-c0f6-4dbf-8779-74fa0379ca1c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:21.762615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84042748-c0f6-4dbf-8779-74fa0379ca1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:21.789568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-84042748-c0f6-4dbf-8779-74fa0379ca1c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:21.790790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-84042748-c0f6-4dbf-8779-74fa0379ca1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:21.866675 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 237/484] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:21.888395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4247b3-cd3c-47dd-b99d-803bed2ce32f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:21.993933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4247b3-cd3c-47dd-b99d-803bed2ce32f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:22.028808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455eb95-53de-4885-a3ea-47f93fbe2617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:22.029438 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455eb95-53de-4885-a3ea-47f93fbe2617 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.029715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455eb95-53de-4885-a3ea-47f93fbe2617 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.029985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455eb95-53de-4885-a3ea-47f93fbe2617 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.040639 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455eb95-53de-4885-a3ea-47f93fbe2617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-1455eb95-53de-4885-a3ea-47f93fbe2617 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:22.045364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:22.047454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:22.049444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 248/485] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:22.056388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84042748-c0f6-4dbf-8779-74fa0379ca1c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:22.073636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:22.162094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84042748-c0f6-4dbf-8779-74fa0379ca1c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:22.210486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6db1bf-ffdb-427c-93ba-f11c26d9a420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:22.211110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6db1bf-ffdb-427c-93ba-f11c26d9a420 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.211459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6db1bf-ffdb-427c-93ba-f11c26d9a420 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.211733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6db1bf-ffdb-427c-93ba-f11c26d9a420 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.212525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6db1bf-ffdb-427c-93ba-f11c26d9a420 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:22.212831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6db1bf-ffdb-427c-93ba-f11c26d9a420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:22.214209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6db1bf-ffdb-427c-93ba-f11c26d9a420 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:22.227980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 238/486] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:22.233647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455eb95-53de-4885-a3ea-47f93fbe2617 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:22.241863 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bb388-c31d-4062-bbe2-b1293546d495 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.242141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bb388-c31d-4062-bbe2-b1293546d495 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.242398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bb388-c31d-4062-bbe2-b1293546d495 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.244672 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 239/487] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:22.249583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1653-ee76-4c0d-a5ff-33734e2c87fb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.249866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1653-ee76-4c0d-a5ff-33734e2c87fb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.250167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1653-ee76-4c0d-a5ff-33734e2c87fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.257815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:22.285595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e79d1653-ee76-4c0d-a5ff-33734e2c87fb None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:22.286877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e79d1653-ee76-4c0d-a5ff-33734e2c87fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:22.348222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455eb95-53de-4885-a3ea-47f93fbe2617 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:22.370761 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 240/488] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:22.371685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d1653-ee76-4c0d-a5ff-33734e2c87fb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:22.384315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:22.403748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd447ea-1aac-44ff-8a6a-c9f13243e2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:22.405227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd447ea-1aac-44ff-8a6a-c9f13243e2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.405605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd447ea-1aac-44ff-8a6a-c9f13243e2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.405969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd447ea-1aac-44ff-8a6a-c9f13243e2f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.417081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd447ea-1aac-44ff-8a6a-c9f13243e2f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:22.418624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd447ea-1aac-44ff-8a6a-c9f13243e2f4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-5bd447ea-1aac-44ff-8a6a-c9f13243e2f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:22.433419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:22.434527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:22.435749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:22.435749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:22.435749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:22.436900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 249/489] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:22.442326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6b63d-9ce0-4925-a995-b84b3119d916 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.442326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6b63d-9ce0-4925-a995-b84b3119d916 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.442326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6b63d-9ce0-4925-a995-b84b3119d916 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.443882 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 250/490] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:22.444662 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6b63d-9ce0-4925-a995-b84b3119d916 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:22.521412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d1653-ee76-4c0d-a5ff-33734e2c87fb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:22.523460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:22.566020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6b63d-9ce0-4925-a995-b84b3119d916 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:22.566572 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d72ef-7f37-47f1-b6ed-3a89e9bc9c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:22.567200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d72ef-7f37-47f1-b6ed-3a89e9bc9c3c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.567474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d72ef-7f37-47f1-b6ed-3a89e9bc9c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.567743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d72ef-7f37-47f1-b6ed-3a89e9bc9c3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.580935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d72ef-7f37-47f1-b6ed-3a89e9bc9c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ed4d72ef-7f37-47f1-b6ed-3a89e9bc9c3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:22.586961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:22.587875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:22.588756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 241/491] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:22.589919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30826ed5-fe67-417b-b3bc-2ed358750e36 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.589919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30826ed5-fe67-417b-b3bc-2ed358750e36 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.590146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30826ed5-fe67-417b-b3bc-2ed358750e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.608755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f045099-90c7-47a4-a416-b2e4bf2dc816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:22.609668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f045099-90c7-47a4-a416-b2e4bf2dc816 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.610196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f045099-90c7-47a4-a416-b2e4bf2dc816 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.610676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f045099-90c7-47a4-a416-b2e4bf2dc816 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.611079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-30826ed5-fe67-417b-b3bc-2ed358750e36 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:22.611764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f045099-90c7-47a4-a416-b2e4bf2dc816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:22.612239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-30826ed5-fe67-417b-b3bc-2ed358750e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:22.612490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f045099-90c7-47a4-a416-b2e4bf2dc816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:22.614180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f045099-90c7-47a4-a416-b2e4bf2dc816 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:22.627562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 251/492] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:22.628936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f045099-90c7-47a4-a416-b2e4bf2dc816 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:22.685661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:22.700418 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 242/493] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:22.700418 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30826ed5-fe67-417b-b3bc-2ed358750e36 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:22.744041 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f045099-90c7-47a4-a416-b2e4bf2dc816 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:22.785040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:22.788653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5ff93-662a-4dd1-b206-f6868cf60dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:22.789655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5ff93-662a-4dd1-b206-f6868cf60dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.790255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5ff93-662a-4dd1-b206-f6868cf60dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.790595 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5ff93-662a-4dd1-b206-f6868cf60dd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.791567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5ff93-662a-4dd1-b206-f6868cf60dd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:22.791961 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5ff93-662a-4dd1-b206-f6868cf60dd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:22.794616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5ff93-662a-4dd1-b206-f6868cf60dd2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:22.804667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 252/494] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:22.806378 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5ff93-662a-4dd1-b206-f6868cf60dd2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:22.834117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30826ed5-fe67-417b-b3bc-2ed358750e36 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:22.881181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76220c04-bf26-437e-adfb-65799a5f3165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:22.882325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76220c04-bf26-437e-adfb-65799a5f3165 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.882744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76220c04-bf26-437e-adfb-65799a5f3165 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.884237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76220c04-bf26-437e-adfb-65799a5f3165 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.893148 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76220c04-bf26-437e-adfb-65799a5f3165 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:22.895362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76220c04-bf26-437e-adfb-65799a5f3165 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-76220c04-bf26-437e-adfb-65799a5f3165 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:22.915107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:22.916240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:22.917409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:22.917409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:22.917409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:22.917409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:22.917409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:22.917409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:22.917409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:22.917409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:22.917409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:22.917409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:22.920003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 243/495] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 30 19:16:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:22.921201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76220c04-bf26-437e-adfb-65799a5f3165 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:22.924790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5ff93-662a-4dd1-b206-f6868cf60dd2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:22.965596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3ca1f-99a2-4e38-b91c-d1ead671d027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:22.966246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3ca1f-99a2-4e38-b91c-d1ead671d027 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:22.966519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3ca1f-99a2-4e38-b91c-d1ead671d027 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:22.967032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3ca1f-99a2-4e38-b91c-d1ead671d027 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:22.980395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3ca1f-99a2-4e38-b91c-d1ead671d027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ecc3ca1f-99a2-4e38-b91c-d1ead671d027 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:22.986104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:22.987059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:22.987872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:22.988545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 253/496] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:22.989247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3ca1f-99a2-4e38-b91c-d1ead671d027 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:22.998057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:23.041450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76220c04-bf26-437e-adfb-65799a5f3165 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:23.051065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:23.077383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48873b14-6fbc-4a0f-9f3a-266d7494dab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:23.078302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48873b14-6fbc-4a0f-9f3a-266d7494dab2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.078628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48873b14-6fbc-4a0f-9f3a-266d7494dab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.078908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48873b14-6fbc-4a0f-9f3a-266d7494dab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.090348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48873b14-6fbc-4a0f-9f3a-266d7494dab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:23.091810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48873b14-6fbc-4a0f-9f3a-266d7494dab2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:23.099220 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3ca1f-99a2-4e38-b91c-d1ead671d027 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-48873b14-6fbc-4a0f-9f3a-266d7494dab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:23.106333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:23.107270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:23.108506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:23.108506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:23.108506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:23.111868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 244/497] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:23.112711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48873b14-6fbc-4a0f-9f3a-266d7494dab2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:23.137646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68193298-9830-4b3f-9a3b-0de7f28c8b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:23.138459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193298-9830-4b3f-9a3b-0de7f28c8b7a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.138870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193298-9830-4b3f-9a3b-0de7f28c8b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.139297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193298-9830-4b3f-9a3b-0de7f28c8b7a None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.140339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68193298-9830-4b3f-9a3b-0de7f28c8b7a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:23.140667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68193298-9830-4b3f-9a3b-0de7f28c8b7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:23.142077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68193298-9830-4b3f-9a3b-0de7f28c8b7a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:23.185844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 254/498] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 30 19:16:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:23.186810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68193298-9830-4b3f-9a3b-0de7f28c8b7a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:23.221916 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48873b14-6fbc-4a0f-9f3a-266d7494dab2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:23.235576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:23.258716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfabde-7174-4024-b56b-b2a7bbac8e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:23.259393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfabde-7174-4024-b56b-b2a7bbac8e40 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.259674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfabde-7174-4024-b56b-b2a7bbac8e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.259943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfabde-7174-4024-b56b-b2a7bbac8e40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.270225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfabde-7174-4024-b56b-b2a7bbac8e40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9ddfabde-7174-4024-b56b-b2a7bbac8e40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:23.276000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:23.276989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:23.277809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 245/499] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:23.278498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfabde-7174-4024-b56b-b2a7bbac8e40 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:23.298448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68193298-9830-4b3f-9a3b-0de7f28c8b7a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:23.311599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:23.333367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb0492-4e37-41b5-a810-79d8c4b1a714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:23.333973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb0492-4e37-41b5-a810-79d8c4b1a714 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.334255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb0492-4e37-41b5-a810-79d8c4b1a714 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.334522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb0492-4e37-41b5-a810-79d8c4b1a714 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.335183 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb0492-4e37-41b5-a810-79d8c4b1a714 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:23.335478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb0492-4e37-41b5-a810-79d8c4b1a714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:23.336924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb0492-4e37-41b5-a810-79d8c4b1a714 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:23.344445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 255/500] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:23.345219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb0492-4e37-41b5-a810-79d8c4b1a714 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:23.375113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfabde-7174-4024-b56b-b2a7bbac8e40 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:23.408220 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bfa73-01ee-445e-9a65-f291d2c97e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:23.408851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bfa73-01ee-445e-9a65-f291d2c97e0a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.409125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bfa73-01ee-445e-9a65-f291d2c97e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.409387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bfa73-01ee-445e-9a65-f291d2c97e0a None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.410016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bfa73-01ee-445e-9a65-f291d2c97e0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:23.410307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bfa73-01ee-445e-9a65-f291d2c97e0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:23.411660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bfa73-01ee-445e-9a65-f291d2c97e0a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:23.423364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 246/501] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:23.424511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033edcac-128b-44ba-9629-ac030fd45bbf None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.424790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033edcac-128b-44ba-9629-ac030fd45bbf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.425055 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033edcac-128b-44ba-9629-ac030fd45bbf None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.426217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 247/502] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:23.426931 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033edcac-128b-44ba-9629-ac030fd45bbf None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:23.447007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb0492-4e37-41b5-a810-79d8c4b1a714 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:23.457422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:23.480491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e250489-d082-4e2e-a208-048f2dd4d7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:23.481091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e250489-d082-4e2e-a208-048f2dd4d7bc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.481365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e250489-d082-4e2e-a208-048f2dd4d7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.481629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e250489-d082-4e2e-a208-048f2dd4d7bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.523817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:23.528876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033edcac-128b-44ba-9629-ac030fd45bbf None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:23.548621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e250489-d082-4e2e-a208-048f2dd4d7bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:23.550019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e250489-d082-4e2e-a208-048f2dd4d7bc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:23.563713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1f9b2-a538-449d-836b-0eb99f868566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:23.564523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f9b2-a538-449d-836b-0eb99f868566 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.564800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f9b2-a538-449d-836b-0eb99f868566 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.565068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f9b2-a538-449d-836b-0eb99f868566 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.565723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1f9b2-a538-449d-836b-0eb99f868566 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:23.566022 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1f9b2-a538-449d-836b-0eb99f868566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:23.567376 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1f9b2-a538-449d-836b-0eb99f868566 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:23.575961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 248/503] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:23.576784 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1f9b2-a538-449d-836b-0eb99f868566 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:23.650560 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 256/504] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:23.651291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e250489-d082-4e2e-a208-048f2dd4d7bc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:23.670966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:23.677793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1f9b2-a538-449d-836b-0eb99f868566 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:23.710574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fc49a-9891-4b66-81e0-7fd9e87b6a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:23.711191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fc49a-9891-4b66-81e0-7fd9e87b6a9b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.711489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fc49a-9891-4b66-81e0-7fd9e87b6a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.711757 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fc49a-9891-4b66-81e0-7fd9e87b6a9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.721835 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fc49a-9891-4b66-81e0-7fd9e87b6a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:23.723250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fc49a-9891-4b66-81e0-7fd9e87b6a9b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:23.724140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7c7fc49a-9891-4b66-81e0-7fd9e87b6a9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:23.738728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:23.739750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:23.740825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:23.740825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:23.740825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:23.740825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 249/505] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:23.741099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788698d1-8ae0-4d63-a679-2c10a64b8cba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.741384 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788698d1-8ae0-4d63-a679-2c10a64b8cba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.741659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788698d1-8ae0-4d63-a679-2c10a64b8cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.753791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e250489-d082-4e2e-a208-048f2dd4d7bc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:23.759472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-788698d1-8ae0-4d63-a679-2c10a64b8cba None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:23.760686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-788698d1-8ae0-4d63-a679-2c10a64b8cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:23.785535 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc31b09-f9a4-4d7c-b11c-03d1be5f1dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:23.786154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc31b09-f9a4-4d7c-b11c-03d1be5f1dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.786430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc31b09-f9a4-4d7c-b11c-03d1be5f1dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.786696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc31b09-f9a4-4d7c-b11c-03d1be5f1dd9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.794140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc31b09-f9a4-4d7c-b11c-03d1be5f1dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:23.795705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc31b09-f9a4-4d7c-b11c-03d1be5f1dd9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-1dc31b09-f9a4-4d7c-b11c-03d1be5f1dd9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:23.810613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:23.811721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:23.812674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:23.812674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:23.812674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:23.812674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:23.812674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:23.812674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:23.812674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:23.812674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:23.812674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 19:16:23.812674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:23.816283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 257/506] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:23.817039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc31b09-f9a4-4d7c-b11c-03d1be5f1dd9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:23.830865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 250/507] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:23.831693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788698d1-8ae0-4d63-a679-2c10a64b8cba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:23.909232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:23.933080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788698d1-8ae0-4d63-a679-2c10a64b8cba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:23.936067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc31b09-f9a4-4d7c-b11c-03d1be5f1dd9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:23.967458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd5e3b-565d-47db-b765-344cf2cda412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:23.968080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd5e3b-565d-47db-b765-344cf2cda412 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.968355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd5e3b-565d-47db-b765-344cf2cda412 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.968650 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd5e3b-565d-47db-b765-344cf2cda412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.970485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b1a7f-9ba0-41fc-a600-079d4e480445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:23.971115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b1a7f-9ba0-41fc-a600-079d4e480445 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:23.971395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b1a7f-9ba0-41fc-a600-079d4e480445 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:23.971668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b1a7f-9ba0-41fc-a600-079d4e480445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:23.973548 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:23.979190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd5e3b-565d-47db-b765-344cf2cda412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:23.980670 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd5e3b-565d-47db-b765-344cf2cda412 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:23.983483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b1a7f-9ba0-41fc-a600-079d4e480445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:23.984952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b1a7f-9ba0-41fc-a600-079d4e480445 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a6dd5e3b-565d-47db-b765-344cf2cda412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:23.995892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:23.997111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:23.997982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:23.997982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:23.997982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-5e4b1a7f-9ba0-41fc-a600-079d4e480445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:23.999979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:24.000990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:24.001851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:24.001851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:24.001851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:24.002023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 251/508] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:24.002605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd5e3b-565d-47db-b765-344cf2cda412 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.004639 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 258/509] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:24.005929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b1a7f-9ba0-41fc-a600-079d4e480445 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.104312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd5e3b-565d-47db-b765-344cf2cda412 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:24.113266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b1a7f-9ba0-41fc-a600-079d4e480445 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:24.129596 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:24.138053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919845d-0fda-4580-851a-eabfbdd68c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:24.138652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919845d-0fda-4580-851a-eabfbdd68c80 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:24.138932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919845d-0fda-4580-851a-eabfbdd68c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:24.139204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919845d-0fda-4580-851a-eabfbdd68c80 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:24.139841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919845d-0fda-4580-851a-eabfbdd68c80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:24.140142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919845d-0fda-4580-851a-eabfbdd68c80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:24.141554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919845d-0fda-4580-851a-eabfbdd68c80 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:24.146456 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805e6e1-430c-4dea-8fb3-69a30d4932a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:24.147051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805e6e1-430c-4dea-8fb3-69a30d4932a8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:24.147325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805e6e1-430c-4dea-8fb3-69a30d4932a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:24.147592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805e6e1-430c-4dea-8fb3-69a30d4932a8 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:24.148303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805e6e1-430c-4dea-8fb3-69a30d4932a8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:24.148632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805e6e1-430c-4dea-8fb3-69a30d4932a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:24.150110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805e6e1-430c-4dea-8fb3-69a30d4932a8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:24.153199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 252/510] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:24.153975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919845d-0fda-4580-851a-eabfbdd68c80 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.168331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 259/511] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:24.168331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805e6e1-430c-4dea-8fb3-69a30d4932a8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.213445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:24.254808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919845d-0fda-4580-851a-eabfbdd68c80 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:24.267816 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805e6e1-430c-4dea-8fb3-69a30d4932a8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:24.288144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a679d-5651-46b4-9a64-506dc78f4820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:24.288769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a679d-5651-46b4-9a64-506dc78f4820 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:24.289041 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a679d-5651-46b4-9a64-506dc78f4820 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:24.289304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a679d-5651-46b4-9a64-506dc78f4820 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:24.290565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a679d-5651-46b4-9a64-506dc78f4820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:24.290858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a679d-5651-46b4-9a64-506dc78f4820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:24.292212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a679d-5651-46b4-9a64-506dc78f4820 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:24.301118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c05c48-177b-4a77-b62b-e95df58d954e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:24.301721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c05c48-177b-4a77-b62b-e95df58d954e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:24.301996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c05c48-177b-4a77-b62b-e95df58d954e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:24.302265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c05c48-177b-4a77-b62b-e95df58d954e None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:24.302868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 253/512] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:24.302988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c05c48-177b-4a77-b62b-e95df58d954e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:24.303189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c05c48-177b-4a77-b62b-e95df58d954e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:24.303629 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a679d-5651-46b4-9a64-506dc78f4820 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.304668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c05c48-177b-4a77-b62b-e95df58d954e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:24.317374 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 260/513] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:24.318899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c05c48-177b-4a77-b62b-e95df58d954e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.350350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:24.400501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a679d-5651-46b4-9a64-506dc78f4820 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:24.418298 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c05c48-177b-4a77-b62b-e95df58d954e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:24.420923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:24.434307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02f1d4-3b37-4c1f-bc2f-f2fee31a8e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:24.434924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02f1d4-3b37-4c1f-bc2f-f2fee31a8e0d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:24.435201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02f1d4-3b37-4c1f-bc2f-f2fee31a8e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:24.435468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02f1d4-3b37-4c1f-bc2f-f2fee31a8e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:24.445619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02f1d4-3b37-4c1f-bc2f-f2fee31a8e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:24.447063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02f1d4-3b37-4c1f-bc2f-f2fee31a8e0d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:24.451412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431437a-a816-40d0-a50f-b9c7371df6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:24.452031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431437a-a816-40d0-a50f-b9c7371df6ac None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:24.452303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431437a-a816-40d0-a50f-b9c7371df6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:24.452601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431437a-a816-40d0-a50f-b9c7371df6ac None admin] PATH_INFO: `/v3/users/glance` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:24.459651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431437a-a816-40d0-a50f-b9c7371df6ac None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bc02f1d4-3b37-4c1f-bc2f-f2fee31a8e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:24.460662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:24.461731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:24.463109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:24.463109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:24.463109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:24.463284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431437a-a816-40d0-a50f-b9c7371df6ac None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:24.463363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 254/514] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:24.464444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02f1d4-3b37-4c1f-bc2f-f2fee31a8e0d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-2431437a-a816-40d0-a50f-b9c7371df6ac None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:24.472403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:24.473319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 261/515] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:24.474225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431437a-a816-40d0-a50f-b9c7371df6ac None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.561658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:24.564915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02f1d4-3b37-4c1f-bc2f-f2fee31a8e0d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:24.571958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431437a-a816-40d0-a50f-b9c7371df6ac None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:24.598483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2485908-291a-4a10-9cbd-ffa8ca2253d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:24.599069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2485908-291a-4a10-9cbd-ffa8ca2253d6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:24.599341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2485908-291a-4a10-9cbd-ffa8ca2253d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:24.599608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2485908-291a-4a10-9cbd-ffa8ca2253d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:24.605544 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d17fe4-9715-4eb4-8bc7-e82fda4c2027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:24.606165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d17fe4-9715-4eb4-8bc7-e82fda4c2027 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:24.606444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d17fe4-9715-4eb4-8bc7-e82fda4c2027 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:24.606712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d17fe4-9715-4eb4-8bc7-e82fda4c2027 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:24.614592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d17fe4-9715-4eb4-8bc7-e82fda4c2027 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:24.616029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d17fe4-9715-4eb4-8bc7-e82fda4c2027 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d3d17fe4-9715-4eb4-8bc7-e82fda4c2027 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:24.628158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:24.629068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:24.629940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:24.629940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:24.629940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:24.629940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:24.629940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:24.629940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:24.629940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:24.629940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:24.629940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 262/516] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:24.630438 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d17fe4-9715-4eb4-8bc7-e82fda4c2027 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.654385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:24.669793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2485908-291a-4a10-9cbd-ffa8ca2253d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:24.671239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2485908-291a-4a10-9cbd-ffa8ca2253d6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:24.726419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d17fe4-9715-4eb4-8bc7-e82fda4c2027 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:24.759302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7379b-57b7-4f96-9406-e778dad6b806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:24.759914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7379b-57b7-4f96-9406-e778dad6b806 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:24.760182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7379b-57b7-4f96-9406-e778dad6b806 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:24.760474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7379b-57b7-4f96-9406-e778dad6b806 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:24.765670 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:24.767799 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7379b-57b7-4f96-9406-e778dad6b806 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:24.769420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7379b-57b7-4f96-9406-e778dad6b806 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:24.773897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 255/517] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:24.774911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2485908-291a-4a10-9cbd-ffa8ca2253d6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-26a7379b-57b7-4f96-9406-e778dad6b806 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:24.780333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:24.781250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 263/518] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:24.782156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7379b-57b7-4f96-9406-e778dad6b806 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.867336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:24.872360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2485908-291a-4a10-9cbd-ffa8ca2253d6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:24.884889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7379b-57b7-4f96-9406-e778dad6b806 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:24.905283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d1fcc-1d71-43d4-b0a5-51d27836f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:24.905917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d1fcc-1d71-43d4-b0a5-51d27836f36e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:24.906191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d1fcc-1d71-43d4-b0a5-51d27836f36e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:24.906454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d1fcc-1d71-43d4-b0a5-51d27836f36e None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:24.907082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d1fcc-1d71-43d4-b0a5-51d27836f36e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:24.907373 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d1fcc-1d71-43d4-b0a5-51d27836f36e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:24.908760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d1fcc-1d71-43d4-b0a5-51d27836f36e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:24.917331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a4aff-5b3c-4682-9b75-171bb4e9bfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:24.917913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a4aff-5b3c-4682-9b75-171bb4e9bfbd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:24.918192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a4aff-5b3c-4682-9b75-171bb4e9bfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:24.918457 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a4aff-5b3c-4682-9b75-171bb4e9bfbd None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:24.919162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a4aff-5b3c-4682-9b75-171bb4e9bfbd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:24.919460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a4aff-5b3c-4682-9b75-171bb4e9bfbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:24.919909 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 256/519] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:24.920714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d1fcc-1d71-43d4-b0a5-51d27836f36e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.920893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a4aff-5b3c-4682-9b75-171bb4e9bfbd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:24.940777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 264/520] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:24.941574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a4aff-5b3c-4682-9b75-171bb4e9bfbd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:24.977841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:25.019007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d1fcc-1d71-43d4-b0a5-51d27836f36e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:25.036465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a4aff-5b3c-4682-9b75-171bb4e9bfbd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:25.051594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3b80c-44f8-4133-b35e-78b1e8cc17bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:25.052183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3b80c-44f8-4133-b35e-78b1e8cc17bc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:25.052485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3b80c-44f8-4133-b35e-78b1e8cc17bc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:25.052755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3b80c-44f8-4133-b35e-78b1e8cc17bc None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:25.053503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3b80c-44f8-4133-b35e-78b1e8cc17bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:25.053801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3b80c-44f8-4133-b35e-78b1e8cc17bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:25.055765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3b80c-44f8-4133-b35e-78b1e8cc17bc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:25.070758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bfce7-2fa9-418f-95e3-a5819ccca313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:25.071403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bfce7-2fa9-418f-95e3-a5819ccca313 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:25.071677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bfce7-2fa9-418f-95e3-a5819ccca313 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:25.071941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bfce7-2fa9-418f-95e3-a5819ccca313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:25.072706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 257/521] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:25.073482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3b80c-44f8-4133-b35e-78b1e8cc17bc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:25.082986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bfce7-2fa9-418f-95e3-a5819ccca313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:25.084468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bfce7-2fa9-418f-95e3-a5819ccca313 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:25.086738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a74bfce7-2fa9-418f-95e3-a5819ccca313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:25.100522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:25.101478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:25.102963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:25.102963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:25.102963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:25.102963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 265/522] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:25.102963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bfce7-2fa9-418f-95e3-a5819ccca313 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:25.174371 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3b80c-44f8-4133-b35e-78b1e8cc17bc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:25.194965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:25.201670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bfce7-2fa9-418f-95e3-a5819ccca313 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:25.209439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf35ba-8fd9-4f76-8937-c75df7ef9215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:25.210033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf35ba-8fd9-4f76-8937-c75df7ef9215 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:25.210304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf35ba-8fd9-4f76-8937-c75df7ef9215 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:25.210567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf35ba-8fd9-4f76-8937-c75df7ef9215 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:25.211332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf35ba-8fd9-4f76-8937-c75df7ef9215 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:25.212131 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf35ba-8fd9-4f76-8937-c75df7ef9215 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:25.213583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf35ba-8fd9-4f76-8937-c75df7ef9215 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:25.233700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c085e3-a1d6-489d-bf53-12ede0d78084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:25.234289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c085e3-a1d6-489d-bf53-12ede0d78084 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:25.234560 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c085e3-a1d6-489d-bf53-12ede0d78084 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:25.234824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c085e3-a1d6-489d-bf53-12ede0d78084 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:25.235447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c085e3-a1d6-489d-bf53-12ede0d78084 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:25.236876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c085e3-a1d6-489d-bf53-12ede0d78084 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:25.248968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 258/523] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Sep 30 19:16:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:25.250120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf35ba-8fd9-4f76-8937-c75df7ef9215 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:25.260401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 266/524] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:16:25 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:16:25.260897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c085e3-a1d6-489d-bf53-12ede0d78084 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:25.331622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:25.362465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf35ba-8fd9-4f76-8937-c75df7ef9215 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:25.399583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c085e3-a1d6-489d-bf53-12ede0d78084 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:25.399772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f81f5-6152-40ec-841b-e6ed1fbcf4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:25.400335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f81f5-6152-40ec-841b-e6ed1fbcf4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:25.400335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f81f5-6152-40ec-841b-e6ed1fbcf4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:25.400537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f81f5-6152-40ec-841b-e6ed1fbcf4b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:25.437223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e04b6-5444-4336-a84d-c29d00718200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:25.438567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e04b6-5444-4336-a84d-c29d00718200 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:25.438567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e04b6-5444-4336-a84d-c29d00718200 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:25.438567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e04b6-5444-4336-a84d-c29d00718200 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:25.439065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e04b6-5444-4336-a84d-c29d00718200 None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=5952db29e6fc4ddbb9aeb0522497312c) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:25.439367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e04b6-5444-4336-a84d-c29d00718200 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:25.440873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e04b6-5444-4336-a84d-c29d00718200 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:25.447747 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 267/525] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:25 2022] GET /identity/v3/role_assignments?user.id=5952db29e6fc4ddbb9aeb0522497312c&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:25.448639 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e04b6-5444-4336-a84d-c29d00718200 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:25.452131 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:25.479433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f81f5-6152-40ec-841b-e6ed1fbcf4b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:25.481402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f81f5-6152-40ec-841b-e6ed1fbcf4b3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:25.594325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e04b6-5444-4336-a84d-c29d00718200 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:25.612002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 259/526] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:25.613007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f81f5-6152-40ec-841b-e6ed1fbcf4b3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:25.615610 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:25.641238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45225f-5d95-4d6c-8f1e-01a9655eecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:25.641238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45225f-5d95-4d6c-8f1e-01a9655eecff None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:25.641498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45225f-5d95-4d6c-8f1e-01a9655eecff None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:25.641612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45225f-5d95-4d6c-8f1e-01a9655eecff None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:25.642254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45225f-5d95-4d6c-8f1e-01a9655eecff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:25.642549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45225f-5d95-4d6c-8f1e-01a9655eecff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:25.643909 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45225f-5d95-4d6c-8f1e-01a9655eecff None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:25.657310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 268/527] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:25.658144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45225f-5d95-4d6c-8f1e-01a9655eecff None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:25.704509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:25.730153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f81f5-6152-40ec-841b-e6ed1fbcf4b3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:25.758749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45225f-5d95-4d6c-8f1e-01a9655eecff None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:25.767866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dd613-4091-4454-9591-953537ba61f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:25.768854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dd613-4091-4454-9591-953537ba61f1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:25.769367 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dd613-4091-4454-9591-953537ba61f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:25.769926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dd613-4091-4454-9591-953537ba61f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:25.797777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ec361-f342-4e51-a2c1-615b365637e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:25.798407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ec361-f342-4e51-a2c1-615b365637e3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:25.798679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ec361-f342-4e51-a2c1-615b365637e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:25.799449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ec361-f342-4e51-a2c1-615b365637e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:25.813228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ec361-f342-4e51-a2c1-615b365637e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:25.815943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ec361-f342-4e51-a2c1-615b365637e3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-210ec361-f342-4e51-a2c1-615b365637e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:25.834566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:25.835564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:25.836509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:25.836509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:25.836509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:25.840506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 269/528] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:25.841407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ec361-f342-4e51-a2c1-615b365637e3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:25.862406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dd613-4091-4454-9591-953537ba61f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:25.863871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dd613-4091-4454-9591-953537ba61f1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:25.880676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:25.960513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:25.975178 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ec361-f342-4e51-a2c1-615b365637e3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:25.992451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 260/529] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:25.992451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dd613-4091-4454-9591-953537ba61f1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:26.019446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca69efa-a137-4137-b3b7-2534ba567534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:26.020509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69efa-a137-4137-b3b7-2534ba567534 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:26.020911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69efa-a137-4137-b3b7-2534ba567534 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:26.021653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69efa-a137-4137-b3b7-2534ba567534 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:26.030275 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca69efa-a137-4137-b3b7-2534ba567534 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:26.032267 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca69efa-a137-4137-b3b7-2534ba567534 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-8ca69efa-a137-4137-b3b7-2534ba567534 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:26.047289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:26.048179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:26.049096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:26.049096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:26.049096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:26.049096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:26.049096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:26.049096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:26.049096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:26.049096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:26.049096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 270/530] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:26.049565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca69efa-a137-4137-b3b7-2534ba567534 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:26.124402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dd613-4091-4454-9591-953537ba61f1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:26.156037 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:26.163064 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a4cd9-50a1-45c6-a7ff-6f66ae028836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:26.163826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a4cd9-50a1-45c6-a7ff-6f66ae028836 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:26.164110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a4cd9-50a1-45c6-a7ff-6f66ae028836 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:26.164984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a4cd9-50a1-45c6-a7ff-6f66ae028836 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:26.165637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a4cd9-50a1-45c6-a7ff-6f66ae028836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:26.165940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a4cd9-50a1-45c6-a7ff-6f66ae028836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:26.167341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a4cd9-50a1-45c6-a7ff-6f66ae028836 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:26.174375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca69efa-a137-4137-b3b7-2534ba567534 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:26.179547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 261/531] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:26.180609 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a4cd9-50a1-45c6-a7ff-6f66ae028836 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:26.216476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d337-8f1d-4ceb-a313-7d6fec79d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:26.217295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d337-8f1d-4ceb-a313-7d6fec79d999 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:26.217572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d337-8f1d-4ceb-a313-7d6fec79d999 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:26.218733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d337-8f1d-4ceb-a313-7d6fec79d999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:26.223716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:26.233068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332d337-8f1d-4ceb-a313-7d6fec79d999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:26.234738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332d337-8f1d-4ceb-a313-7d6fec79d999 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-7332d337-8f1d-4ceb-a313-7d6fec79d999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:26.252930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:26.254077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:26.255138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:26.255138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:26.255138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:26.257389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 271/532] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:26.258385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d337-8f1d-4ceb-a313-7d6fec79d999 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:26.292006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a4cd9-50a1-45c6-a7ff-6f66ae028836 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:26.329916 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cc9c9-641d-44ee-8779-f4057c0ed7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:26.330731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cc9c9-641d-44ee-8779-f4057c0ed7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:26.331112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cc9c9-641d-44ee-8779-f4057c0ed7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:26.331484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cc9c9-641d-44ee-8779-f4057c0ed7a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:26.332278 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cc9c9-641d-44ee-8779-f4057c0ed7a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:26.332846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cc9c9-641d-44ee-8779-f4057c0ed7a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:26.334399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cc9c9-641d-44ee-8779-f4057c0ed7a6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:26.353016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 262/533] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:26.353016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cc9c9-641d-44ee-8779-f4057c0ed7a6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:26.385485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d337-8f1d-4ceb-a313-7d6fec79d999 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:26.428214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33de853-b0db-4fe8-89a3-049b0dfb5571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:26.429241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33de853-b0db-4fe8-89a3-049b0dfb5571 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:26.429752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33de853-b0db-4fe8-89a3-049b0dfb5571 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:26.430186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33de853-b0db-4fe8-89a3-049b0dfb5571 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:26.438373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33de853-b0db-4fe8-89a3-049b0dfb5571 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:26.439940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33de853-b0db-4fe8-89a3-049b0dfb5571 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:26.441738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a33de853-b0db-4fe8-89a3-049b0dfb5571 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:26.453397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:26.454277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:26.455159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:26.455159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:26.455159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:26.455159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:26.455159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:26.455159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:26.455159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:26.455159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:26.457554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 272/534] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:26.458591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33de853-b0db-4fe8-89a3-049b0dfb5571 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:26.472414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:26.477325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cc9c9-641d-44ee-8779-f4057c0ed7a6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:26.514613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4290d2d-03b4-4370-bd74-4a65d8706759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:26.515445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4290d2d-03b4-4370-bd74-4a65d8706759 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:26.515826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4290d2d-03b4-4370-bd74-4a65d8706759 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:26.516437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4290d2d-03b4-4370-bd74-4a65d8706759 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:26.517071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4290d2d-03b4-4370-bd74-4a65d8706759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:26.517494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4290d2d-03b4-4370-bd74-4a65d8706759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:26.518967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4290d2d-03b4-4370-bd74-4a65d8706759 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:26.548454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 263/535] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:26.549516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4290d2d-03b4-4370-bd74-4a65d8706759 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:26.576808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33de853-b0db-4fe8-89a3-049b0dfb5571 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:26.617707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d793ddd-24bb-4d78-a5a7-c3b878eb0d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:26.618362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d793ddd-24bb-4d78-a5a7-c3b878eb0d70 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:26.618625 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d793ddd-24bb-4d78-a5a7-c3b878eb0d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:26.618898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d793ddd-24bb-4d78-a5a7-c3b878eb0d70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:26.619617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d793ddd-24bb-4d78-a5a7-c3b878eb0d70 None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=65d3020e1f6c414caabbaa185fb7ed48) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:26.619918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d793ddd-24bb-4d78-a5a7-c3b878eb0d70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:26.622238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d793ddd-24bb-4d78-a5a7-c3b878eb0d70 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:26.630581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 273/536] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:26 2022] GET /identity/v3/role_assignments?user.id=65d3020e1f6c414caabbaa185fb7ed48&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:26.631888 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d793ddd-24bb-4d78-a5a7-c3b878eb0d70 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:26.707763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:26.708084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4290d2d-03b4-4370-bd74-4a65d8706759 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:26.757828 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d793ddd-24bb-4d78-a5a7-c3b878eb0d70 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:26.778460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020867e-f87d-4d02-adb4-9adbec13fab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:26.779462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020867e-f87d-4d02-adb4-9adbec13fab4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:26.779968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020867e-f87d-4d02-adb4-9adbec13fab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:26.782530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020867e-f87d-4d02-adb4-9adbec13fab4 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:26.783547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020867e-f87d-4d02-adb4-9adbec13fab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:26.784067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020867e-f87d-4d02-adb4-9adbec13fab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:26.785877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020867e-f87d-4d02-adb4-9adbec13fab4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:26.815052 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 264/537] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:26.821337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020867e-f87d-4d02-adb4-9adbec13fab4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:26.842375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cc0c3-068f-412b-beb8-891217e16e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:26.843449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cc0c3-068f-412b-beb8-891217e16e27 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:26.844041 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cc0c3-068f-412b-beb8-891217e16e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:26.844718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cc0c3-068f-412b-beb8-891217e16e27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:26.861183 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cc0c3-068f-412b-beb8-891217e16e27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:26.864178 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:26.865028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cc0c3-068f-412b-beb8-891217e16e27 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-e00cc0c3-068f-412b-beb8-891217e16e27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:26.887371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:26.888295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:26.889208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:26.889208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:26.889208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:26.889208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:26.889208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:26.889208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:26.889208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:26.889208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:26.899280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 274/538] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:26.909228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cc0c3-068f-412b-beb8-891217e16e27 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:27.019306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cc0c3-068f-412b-beb8-891217e16e27 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:27.044078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:27.059803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f53a1-d1ee-4816-9deb-067c64a41fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:27.060980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f53a1-d1ee-4816-9deb-067c64a41fee None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:27.061487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f53a1-d1ee-4816-9deb-067c64a41fee None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:27.061974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f53a1-d1ee-4816-9deb-067c64a41fee None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:27.062402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020867e-f87d-4d02-adb4-9adbec13fab4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:27.063286 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f53a1-d1ee-4816-9deb-067c64a41fee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:27.063790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f53a1-d1ee-4816-9deb-067c64a41fee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:27.066915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f53a1-d1ee-4816-9deb-067c64a41fee None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:27.087599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 275/539] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:27.093725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f53a1-d1ee-4816-9deb-067c64a41fee None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:27.143482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039525ab-eb23-477c-b000-ba9eed2d27a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:27.144146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039525ab-eb23-477c-b000-ba9eed2d27a8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:27.144462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039525ab-eb23-477c-b000-ba9eed2d27a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:27.144753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039525ab-eb23-477c-b000-ba9eed2d27a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:27.145473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039525ab-eb23-477c-b000-ba9eed2d27a8 None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=58a9b371434243daa4176215daefb79c) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:27.145820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039525ab-eb23-477c-b000-ba9eed2d27a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:27.147260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039525ab-eb23-477c-b000-ba9eed2d27a8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:27.153938 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 265/540] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:26 2022] GET /identity/v3/role_assignments?user.id=58a9b371434243daa4176215daefb79c&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:27.208793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f53a1-d1ee-4816-9deb-067c64a41fee None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:27.248646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d846b-98d8-4fb9-b801-eecd9fe34050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:27.249772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d846b-98d8-4fb9-b801-eecd9fe34050 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:27.250260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d846b-98d8-4fb9-b801-eecd9fe34050 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:27.250737 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d846b-98d8-4fb9-b801-eecd9fe34050 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:27.251707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d846b-98d8-4fb9-b801-eecd9fe34050 None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=18ea49d1f6c040f18511535b7e51a53b) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:27.252229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d846b-98d8-4fb9-b801-eecd9fe34050 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:27.253944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d846b-98d8-4fb9-b801-eecd9fe34050 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:27.261474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 276/541] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:27 2022] GET /identity/v3/role_assignments?user.id=18ea49d1f6c040f18511535b7e51a53b&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:27.707633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a7797-7bba-459d-9834-af1349eabfd9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:27.707964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a7797-7bba-459d-9834-af1349eabfd9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:27.708302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a7797-7bba-459d-9834-af1349eabfd9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:27.709953 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 266/542] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:27.716229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389306-fb8a-49b5-8a4b-8dc8e84e266c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:27.716679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389306-fb8a-49b5-8a4b-8dc8e84e266c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:27.716858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea389306-fb8a-49b5-8a4b-8dc8e84e266c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:27.741339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ea389306-fb8a-49b5-8a4b-8dc8e84e266c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:27.743088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ea389306-fb8a-49b5-8a4b-8dc8e84e266c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:27.810932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:27.844958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 277/543] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 19:16:27.914088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a222ba-d117-45dc-9633-0a50580946d9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:27.914482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a222ba-d117-45dc-9633-0a50580946d9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:27.914707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a222ba-d117-45dc-9633-0a50580946d9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:27.916327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 267/544] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:27.921162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054106f6-c6ff-49ad-a090-478b3a3a3331 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:27.921686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054106f6-c6ff-49ad-a090-478b3a3a3331 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:27.922161 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054106f6-c6ff-49ad-a090-478b3a3a3331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:27.944303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-054106f6-c6ff-49ad-a090-478b3a3a3331 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:27.945848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-054106f6-c6ff-49ad-a090-478b3a3a3331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:28.036219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 278/545] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:28.185109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a222ba-d117-45dc-9633-0a50580946d9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:28.290409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:28.311413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a222ba-d117-45dc-9633-0a50580946d9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:28.352575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e599e-5486-44d4-aa1a-930bcddc7c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:28.353206 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e599e-5486-44d4-aa1a-930bcddc7c63 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:28.353487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e599e-5486-44d4-aa1a-930bcddc7c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:28.353760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e599e-5486-44d4-aa1a-930bcddc7c63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:28.366617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e599e-5486-44d4-aa1a-930bcddc7c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-228e599e-5486-44d4-aa1a-930bcddc7c63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:28.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:28.372625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:28.373458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 268/546] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:28.378725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054106f6-c6ff-49ad-a090-478b3a3a3331 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:28.515457 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054106f6-c6ff-49ad-a090-478b3a3a3331 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:28.522966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:28.547636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e599e-5486-44d4-aa1a-930bcddc7c63 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:28.558874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80c906-6b47-4014-ae07-03eee3374653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:28.559759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80c906-6b47-4014-ae07-03eee3374653 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:28.560254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80c906-6b47-4014-ae07-03eee3374653 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:28.560740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80c906-6b47-4014-ae07-03eee3374653 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:28.561593 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80c906-6b47-4014-ae07-03eee3374653 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:28.562087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80c906-6b47-4014-ae07-03eee3374653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:28.563666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80c906-6b47-4014-ae07-03eee3374653 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:28.583190 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 279/547] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:28.586809 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80c906-6b47-4014-ae07-03eee3374653 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:28.729998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80c906-6b47-4014-ae07-03eee3374653 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:28.739966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e599e-5486-44d4-aa1a-930bcddc7c63 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:28.765784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ff065-7797-4d96-a61c-90ff34e56177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:28.766563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ff065-7797-4d96-a61c-90ff34e56177 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:28.766978 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ff065-7797-4d96-a61c-90ff34e56177 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:28.767345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ff065-7797-4d96-a61c-90ff34e56177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:28.783197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ff065-7797-4d96-a61c-90ff34e56177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:28.784688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ff065-7797-4d96-a61c-90ff34e56177 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-5d8ff065-7797-4d96-a61c-90ff34e56177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:28.806205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:28.807136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:28.808012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:28.808012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:28.808012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:28.808789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 280/548] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:28.812494 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ff065-7797-4d96-a61c-90ff34e56177 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:28.817718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a247ea8-f91d-4e5a-b31f-cc50c9ad750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:28.818349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a247ea8-f91d-4e5a-b31f-cc50c9ad750b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:28.818622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a247ea8-f91d-4e5a-b31f-cc50c9ad750b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:28.818888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a247ea8-f91d-4e5a-b31f-cc50c9ad750b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:28.838695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a247ea8-f91d-4e5a-b31f-cc50c9ad750b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5a247ea8-f91d-4e5a-b31f-cc50c9ad750b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:28.843795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:28.845245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:28.846259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:28.846917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 269/549] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:28.852733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a247ea8-f91d-4e5a-b31f-cc50c9ad750b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:28.859273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:28.869386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:29.006895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ff065-7797-4d96-a61c-90ff34e56177 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:29.028218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a247ea8-f91d-4e5a-b31f-cc50c9ad750b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:29.073570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce850c-2dfc-4ef2-9746-c4bc647bb448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:29.074111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce850c-2dfc-4ef2-9746-c4bc647bb448 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.074394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce850c-2dfc-4ef2-9746-c4bc647bb448 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.074667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce850c-2dfc-4ef2-9746-c4bc647bb448 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.075308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce850c-2dfc-4ef2-9746-c4bc647bb448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:29.075609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce850c-2dfc-4ef2-9746-c4bc647bb448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:29.077001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce850c-2dfc-4ef2-9746-c4bc647bb448 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:29.098351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a8401-bc4b-44c3-a8ab-b739c74e6399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:29.098963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a8401-bc4b-44c3-a8ab-b739c74e6399 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.099239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a8401-bc4b-44c3-a8ab-b739c74e6399 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.099513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a8401-bc4b-44c3-a8ab-b739c74e6399 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.100180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a8401-bc4b-44c3-a8ab-b739c74e6399 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:29.100508 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a8401-bc4b-44c3-a8ab-b739c74e6399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:29.102822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a8401-bc4b-44c3-a8ab-b739c74e6399 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:29.106143 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 281/550] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:29.112193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce850c-2dfc-4ef2-9746-c4bc647bb448 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:29.120580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 270/551] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:29.124024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a8401-bc4b-44c3-a8ab-b739c74e6399 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:29.212825 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:29.230626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce850c-2dfc-4ef2-9746-c4bc647bb448 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:29.274384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bd252-b5da-4db7-b19e-75f7f282432b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:29.275043 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bd252-b5da-4db7-b19e-75f7f282432b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.275324 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bd252-b5da-4db7-b19e-75f7f282432b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.275592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bd252-b5da-4db7-b19e-75f7f282432b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.289556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057bd252-b5da-4db7-b19e-75f7f282432b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:29.291729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057bd252-b5da-4db7-b19e-75f7f282432b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:29.297746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-057bd252-b5da-4db7-b19e-75f7f282432b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:29.324589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:29.325746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:29.326713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:29.326713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:29.326713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:29.326713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:29.326713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:29.326713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:29.326713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:29.326713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:29.326713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:29.326713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:29.327816 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 282/552] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:29.333841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518e201-b8ad-47a8-8cff-8896ea5b5013 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.334546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518e201-b8ad-47a8-8cff-8896ea5b5013 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.334827 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518e201-b8ad-47a8-8cff-8896ea5b5013 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.337305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 283/553] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:16:29.338019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518e201-b8ad-47a8-8cff-8896ea5b5013 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:29.359116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a8401-bc4b-44c3-a8ab-b739c74e6399 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:29.406982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72612c91-7c5b-4a69-85a9-ab8577a84a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:29.407725 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72612c91-7c5b-4a69-85a9-ab8577a84a57 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.408222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72612c91-7c5b-4a69-85a9-ab8577a84a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.408643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72612c91-7c5b-4a69-85a9-ab8577a84a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.425169 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72612c91-7c5b-4a69-85a9-ab8577a84a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:29.427369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72612c91-7c5b-4a69-85a9-ab8577a84a57 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-72612c91-7c5b-4a69-85a9-ab8577a84a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:29.447700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:29.449196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:29.450403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:29.450403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:29.450403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:29.454739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 271/554] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:29.456846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaa8be-43fd-413a-afcb-2bb9e6591407 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.457498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaa8be-43fd-413a-afcb-2bb9e6591407 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.457901 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaa8be-43fd-413a-afcb-2bb9e6591407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.467157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518e201-b8ad-47a8-8cff-8896ea5b5013 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:29.482554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cebaa8be-43fd-413a-afcb-2bb9e6591407 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:29.483745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cebaa8be-43fd-413a-afcb-2bb9e6591407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:29.506856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb143-73fd-4597-a6e3-5d2f90cc3702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:29.507936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb143-73fd-4597-a6e3-5d2f90cc3702 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.509034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb143-73fd-4597-a6e3-5d2f90cc3702 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.509572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb143-73fd-4597-a6e3-5d2f90cc3702 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.510503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdb143-73fd-4597-a6e3-5d2f90cc3702 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:29.511059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdb143-73fd-4597-a6e3-5d2f90cc3702 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:29.512915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdb143-73fd-4597-a6e3-5d2f90cc3702 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:29.523539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:29.535027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 284/555] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 30 19:16:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:29.536205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb143-73fd-4597-a6e3-5d2f90cc3702 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:29.568394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 272/556] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:29.568394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaa8be-43fd-413a-afcb-2bb9e6591407 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:29.648092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:29.662913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb143-73fd-4597-a6e3-5d2f90cc3702 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:29.707283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcfcd1-d960-43c3-97c3-cfcf562610e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:29.708306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcfcd1-d960-43c3-97c3-cfcf562610e7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.708824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcfcd1-d960-43c3-97c3-cfcf562610e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.709424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcfcd1-d960-43c3-97c3-cfcf562610e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.710301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcfcd1-d960-43c3-97c3-cfcf562610e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:29.710828 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcfcd1-d960-43c3-97c3-cfcf562610e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:29.712622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcfcd1-d960-43c3-97c3-cfcf562610e7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:29.726366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 285/557] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:29.731882 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcfcd1-d960-43c3-97c3-cfcf562610e7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:29.765275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaa8be-43fd-413a-afcb-2bb9e6591407 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:29.829360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a191a21-9cf0-40ad-8640-b15d346d0bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:29.830336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a191a21-9cf0-40ad-8640-b15d346d0bff None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.830619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a191a21-9cf0-40ad-8640-b15d346d0bff None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.830893 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a191a21-9cf0-40ad-8640-b15d346d0bff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.833800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:29.880530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcfcd1-d960-43c3-97c3-cfcf562610e7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:29.902808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a191a21-9cf0-40ad-8640-b15d346d0bff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:29.904201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a191a21-9cf0-40ad-8640-b15d346d0bff None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:29.916488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0835b-4509-4cd8-bdd9-091a3b812ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:29.917115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0835b-4509-4cd8-bdd9-091a3b812ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.917590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0835b-4509-4cd8-bdd9-091a3b812ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.917758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0835b-4509-4cd8-bdd9-091a3b812ca8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.924899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0835b-4509-4cd8-bdd9-091a3b812ca8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:29.926306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0835b-4509-4cd8-bdd9-091a3b812ca8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f0b0835b-4509-4cd8-bdd9-091a3b812ca8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:29.937584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:29.938651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:29.939644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:29.939644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:29.939644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:29.939644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:29.939644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:29.939644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:29.939644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:29.939644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:29.939644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:29.939644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:29.939644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 286/558] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 30 19:16:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:29.941767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17518c4-7dc8-416f-a622-4b2b6ad601f2 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.942477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17518c4-7dc8-416f-a622-4b2b6ad601f2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.942756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17518c4-7dc8-416f-a622-4b2b6ad601f2 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.944493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 287/559] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:29.946392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313d8cb-b1b4-455c-90fe-ecde6fd531ab None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:29.946673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313d8cb-b1b4-455c-90fe-ecde6fd531ab None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:29.947235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313d8cb-b1b4-455c-90fe-ecde6fd531ab None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:29.948428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 288/560] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:29.949259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313d8cb-b1b4-455c-90fe-ecde6fd531ab None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:29.950490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:30.005276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 273/561] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:30.005943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a191a21-9cf0-40ad-8640-b15d346d0bff None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:30.053866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313d8cb-b1b4-455c-90fe-ecde6fd531ab None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:30.070938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:30.088667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdca7ff-9acd-4281-bdd8-dc0a01de4408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:30.089291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdca7ff-9acd-4281-bdd8-dc0a01de4408 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.089564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdca7ff-9acd-4281-bdd8-dc0a01de4408 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.089834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdca7ff-9acd-4281-bdd8-dc0a01de4408 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.099827 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdca7ff-9acd-4281-bdd8-dc0a01de4408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:30.104469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a191a21-9cf0-40ad-8640-b15d346d0bff None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-8cdca7ff-9acd-4281-bdd8-dc0a01de4408 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:30.105106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:30.106260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 289/562] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:30.107646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e6a87-6a8a-4f5a-90d9-49fdfc4519f7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.108474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e6a87-6a8a-4f5a-90d9-49fdfc4519f7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.108474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e6a87-6a8a-4f5a-90d9-49fdfc4519f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.124844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d29e6a87-6a8a-4f5a-90d9-49fdfc4519f7 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:30.126098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d29e6a87-6a8a-4f5a-90d9-49fdfc4519f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:30.139755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef77d1c-0d2d-43a5-9e0d-c36241a8377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:30.140356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77d1c-0d2d-43a5-9e0d-c36241a8377f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.140652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77d1c-0d2d-43a5-9e0d-c36241a8377f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.140921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77d1c-0d2d-43a5-9e0d-c36241a8377f None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.141635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef77d1c-0d2d-43a5-9e0d-c36241a8377f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:30.141935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef77d1c-0d2d-43a5-9e0d-c36241a8377f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:30.143362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef77d1c-0d2d-43a5-9e0d-c36241a8377f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:30.161427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 274/563] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 30 19:16:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:30.162887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f8f3-c5d6-41a5-a011-f3346ebe7120 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.163177 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f8f3-c5d6-41a5-a011-f3346ebe7120 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.163477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f8f3-c5d6-41a5-a011-f3346ebe7120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.172056 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:30.182582 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-67b3f8f3-c5d6-41a5-a011-f3346ebe7120 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:30.183756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-67b3f8f3-c5d6-41a5-a011-f3346ebe7120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:30.192765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 290/564] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:30.193590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e6a87-6a8a-4f5a-90d9-49fdfc4519f7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:30.265389 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 275/565] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:30.266225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3f8f3-c5d6-41a5-a011-f3346ebe7120 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:30.295605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e6a87-6a8a-4f5a-90d9-49fdfc4519f7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:30.306952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:30.335491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0b201-8efe-4ee9-8fdc-eb416a836843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:30.336860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b201-8efe-4ee9-8fdc-eb416a836843 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.337558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b201-8efe-4ee9-8fdc-eb416a836843 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.337975 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b201-8efe-4ee9-8fdc-eb416a836843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.349353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0b201-8efe-4ee9-8fdc-eb416a836843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:30.351190 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0b201-8efe-4ee9-8fdc-eb416a836843 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-b4f0b201-8efe-4ee9-8fdc-eb416a836843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:30.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:30.369754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:30.370657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:30.370657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:30.370657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:30.372902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 291/566] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:30.374297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0b201-8efe-4ee9-8fdc-eb416a836843 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:30.411356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3f8f3-c5d6-41a5-a011-f3346ebe7120 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:30.454679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06f460-e973-487b-80ba-f2a09fcd7af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:30.455504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06f460-e973-487b-80ba-f2a09fcd7af6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.455923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06f460-e973-487b-80ba-f2a09fcd7af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.456332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06f460-e973-487b-80ba-f2a09fcd7af6 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.457168 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06f460-e973-487b-80ba-f2a09fcd7af6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:30.457631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06f460-e973-487b-80ba-f2a09fcd7af6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:30.459126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06f460-e973-487b-80ba-f2a09fcd7af6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:30.460798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:30.474311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 276/567] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:30.475397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06f460-e973-487b-80ba-f2a09fcd7af6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:30.501779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0b201-8efe-4ee9-8fdc-eb416a836843 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:30.538233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300127f3-37e5-418d-88fa-48d4274ca4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:30.538853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300127f3-37e5-418d-88fa-48d4274ca4bf None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.539133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300127f3-37e5-418d-88fa-48d4274ca4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.539924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300127f3-37e5-418d-88fa-48d4274ca4bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.550447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300127f3-37e5-418d-88fa-48d4274ca4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:30.551897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300127f3-37e5-418d-88fa-48d4274ca4bf None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-300127f3-37e5-418d-88fa-48d4274ca4bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:30.567596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:30.568813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:30.569886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:30.569886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:30.569886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:30.569886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 292/568] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:30.569886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300127f3-37e5-418d-88fa-48d4274ca4bf None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:30.584292 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06f460-e973-487b-80ba-f2a09fcd7af6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:30.585138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:30.619281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c4845-61fd-4e4a-a6ac-746147baafee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:30.619885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c4845-61fd-4e4a-a6ac-746147baafee None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.620167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c4845-61fd-4e4a-a6ac-746147baafee None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.620468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c4845-61fd-4e4a-a6ac-746147baafee None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.621107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c4845-61fd-4e4a-a6ac-746147baafee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:30.621406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c4845-61fd-4e4a-a6ac-746147baafee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:30.622777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c4845-61fd-4e4a-a6ac-746147baafee None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:30.634245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 277/569] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:30.635008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c4845-61fd-4e4a-a6ac-746147baafee None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:30.671472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300127f3-37e5-418d-88fa-48d4274ca4bf None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:30.688025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:30.704865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b976dc-81cf-442a-9bf5-e9d7418a9dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:30.705438 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b976dc-81cf-442a-9bf5-e9d7418a9dcc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.705715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b976dc-81cf-442a-9bf5-e9d7418a9dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.705982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b976dc-81cf-442a-9bf5-e9d7418a9dcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.716034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b976dc-81cf-442a-9bf5-e9d7418a9dcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-e4b976dc-81cf-442a-9bf5-e9d7418a9dcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:30.721219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:30.722247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 293/570] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:30.723101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b976dc-81cf-442a-9bf5-e9d7418a9dcc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:30.734852 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c4845-61fd-4e4a-a6ac-746147baafee None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:30.768769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90807d19-fda4-4fc3-910f-2a6aae43217c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:30.769382 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90807d19-fda4-4fc3-910f-2a6aae43217c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.769640 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90807d19-fda4-4fc3-910f-2a6aae43217c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.769904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90807d19-fda4-4fc3-910f-2a6aae43217c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.780215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90807d19-fda4-4fc3-910f-2a6aae43217c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:30.781682 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90807d19-fda4-4fc3-910f-2a6aae43217c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-90807d19-fda4-4fc3-910f-2a6aae43217c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:30.796407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:30.797516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:30.798616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:30.798616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:30.798616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:30.798616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 278/571] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:30.798616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90807d19-fda4-4fc3-910f-2a6aae43217c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:30.799200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:30.830562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b976dc-81cf-442a-9bf5-e9d7418a9dcc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:30.864135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa7ae9-cd4c-43a0-b7fa-8b1ff182de4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:30.864764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa7ae9-cd4c-43a0-b7fa-8b1ff182de4a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.865044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa7ae9-cd4c-43a0-b7fa-8b1ff182de4a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.865313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa7ae9-cd4c-43a0-b7fa-8b1ff182de4a None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.865947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa7ae9-cd4c-43a0-b7fa-8b1ff182de4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:30.866249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa7ae9-cd4c-43a0-b7fa-8b1ff182de4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:30.867604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa7ae9-cd4c-43a0-b7fa-8b1ff182de4a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:30.878856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 294/572] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:30.879628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa7ae9-cd4c-43a0-b7fa-8b1ff182de4a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:30.895658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90807d19-fda4-4fc3-910f-2a6aae43217c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:30.904665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:30.929019 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cca25-e1bd-48ae-a1e4-17256022a47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:30.929631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cca25-e1bd-48ae-a1e4-17256022a47b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:30.930431 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cca25-e1bd-48ae-a1e4-17256022a47b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:30.930703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cca25-e1bd-48ae-a1e4-17256022a47b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:30.943003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188cca25-e1bd-48ae-a1e4-17256022a47b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-188cca25-e1bd-48ae-a1e4-17256022a47b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:30.948726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:30.949659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 279/573] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:30.950540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cca25-e1bd-48ae-a1e4-17256022a47b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:30.984254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa7ae9-cd4c-43a0-b7fa-8b1ff182de4a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.013226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:31.022954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc5710-fa4e-4301-98b8-0c5c64681ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.024077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc5710-fa4e-4301-98b8-0c5c64681ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.024361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc5710-fa4e-4301-98b8-0c5c64681ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.024656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc5710-fa4e-4301-98b8-0c5c64681ae3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.031084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc5710-fa4e-4301-98b8-0c5c64681ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.032540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc5710-fa4e-4301-98b8-0c5c64681ae3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-abbc5710-fa4e-4301-98b8-0c5c64681ae3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:31.044118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:31.045059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:31.045980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:31.045980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:31.045980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:31.045980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:31.045980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:31.045980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:31.045980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:31.045980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:31.045980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 295/574] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:31.045980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc5710-fa4e-4301-98b8-0c5c64681ae3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.046540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cca25-e1bd-48ae-a1e4-17256022a47b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.081594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a63e49-1fec-42d4-841c-10d0100b70b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.082186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a63e49-1fec-42d4-841c-10d0100b70b5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.082455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a63e49-1fec-42d4-841c-10d0100b70b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.082719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a63e49-1fec-42d4-841c-10d0100b70b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.083363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a63e49-1fec-42d4-841c-10d0100b70b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:31.083654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a63e49-1fec-42d4-841c-10d0100b70b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.085023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a63e49-1fec-42d4-841c-10d0100b70b5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.093076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 280/575] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:31.093920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a63e49-1fec-42d4-841c-10d0100b70b5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.125678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:31.145578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc5710-fa4e-4301-98b8-0c5c64681ae3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.178195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c331fc-4d49-46b5-93ab-131df2c50d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.178787 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c331fc-4d49-46b5-93ab-131df2c50d7f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.179067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c331fc-4d49-46b5-93ab-131df2c50d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.179335 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c331fc-4d49-46b5-93ab-131df2c50d7f None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.180586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c331fc-4d49-46b5-93ab-131df2c50d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:31.180884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c331fc-4d49-46b5-93ab-131df2c50d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.182239 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c331fc-4d49-46b5-93ab-131df2c50d7f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.193271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a63e49-1fec-42d4-841c-10d0100b70b5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.193395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 296/576] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:31.194309 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c331fc-4d49-46b5-93ab-131df2c50d7f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.225861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de6074-e53c-425a-914f-da91d0b20a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.226473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de6074-e53c-425a-914f-da91d0b20a1a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.227262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de6074-e53c-425a-914f-da91d0b20a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.227527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de6074-e53c-425a-914f-da91d0b20a1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.233951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de6074-e53c-425a-914f-da91d0b20a1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.235443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de6074-e53c-425a-914f-da91d0b20a1a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-66de6074-e53c-425a-914f-da91d0b20a1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:31.247536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:31.248633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:31.249756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:31.249756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:31.249756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:31.249756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:31.249756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:31.249756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:31.249756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:31.249756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:31.250649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:31.250826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 281/577] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:31.250826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de6074-e53c-425a-914f-da91d0b20a1a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.297548 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c331fc-4d49-46b5-93ab-131df2c50d7f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.330315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac4bfd-48a2-449a-b3a6-e6c378fe892a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.330932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac4bfd-48a2-449a-b3a6-e6c378fe892a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.331207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac4bfd-48a2-449a-b3a6-e6c378fe892a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.331471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac4bfd-48a2-449a-b3a6-e6c378fe892a None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.332112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac4bfd-48a2-449a-b3a6-e6c378fe892a None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:31.332429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac4bfd-48a2-449a-b3a6-e6c378fe892a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.333765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac4bfd-48a2-449a-b3a6-e6c378fe892a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.337134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:31.342571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 297/578] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:31.343344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac4bfd-48a2-449a-b3a6-e6c378fe892a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.349329 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de6074-e53c-425a-914f-da91d0b20a1a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.382173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1e61b-8cef-483d-97ab-2ea031c3408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.382772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1e61b-8cef-483d-97ab-2ea031c3408f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.383544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1e61b-8cef-483d-97ab-2ea031c3408f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.383814 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1e61b-8cef-483d-97ab-2ea031c3408f None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.384528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1e61b-8cef-483d-97ab-2ea031c3408f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:31.384829 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1e61b-8cef-483d-97ab-2ea031c3408f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.386266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1e61b-8cef-483d-97ab-2ea031c3408f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.399058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 282/579] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:31.399859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1e61b-8cef-483d-97ab-2ea031c3408f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.448783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac4bfd-48a2-449a-b3a6-e6c378fe892a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.461767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:31.482355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4846745-48c3-4a3d-b417-7fd83d9b2fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.482970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4846745-48c3-4a3d-b417-7fd83d9b2fbd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.483243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4846745-48c3-4a3d-b417-7fd83d9b2fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.483507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4846745-48c3-4a3d-b417-7fd83d9b2fbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.494124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4846745-48c3-4a3d-b417-7fd83d9b2fbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.495475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4846745-48c3-4a3d-b417-7fd83d9b2fbd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.495904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1e61b-8cef-483d-97ab-2ea031c3408f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c4846745-48c3-4a3d-b417-7fd83d9b2fbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:31.509053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:31.514583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:31.515783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:31.515783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:31.515783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:31.515783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 298/580] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:31.515783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4846745-48c3-4a3d-b417-7fd83d9b2fbd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.534986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b28d0-9259-409a-bd22-3efb5c3fa5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.535606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b28d0-9259-409a-bd22-3efb5c3fa5fa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.535937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b28d0-9259-409a-bd22-3efb5c3fa5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.536209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b28d0-9259-409a-bd22-3efb5c3fa5fa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.544022 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b28d0-9259-409a-bd22-3efb5c3fa5fa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.545498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b28d0-9259-409a-bd22-3efb5c3fa5fa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-920b28d0-9259-409a-bd22-3efb5c3fa5fa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:31.556812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:31.557948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 283/581] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:31.559090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b28d0-9259-409a-bd22-3efb5c3fa5fa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.570626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:31.610507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4846745-48c3-4a3d-b417-7fd83d9b2fbd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.643087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30ff34-c24d-4bf0-ae87-9b2898a9487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.643677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30ff34-c24d-4bf0-ae87-9b2898a9487e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.643951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30ff34-c24d-4bf0-ae87-9b2898a9487e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.644215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30ff34-c24d-4bf0-ae87-9b2898a9487e None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.644929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30ff34-c24d-4bf0-ae87-9b2898a9487e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:31.645221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30ff34-c24d-4bf0-ae87-9b2898a9487e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.646639 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30ff34-c24d-4bf0-ae87-9b2898a9487e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.656117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b28d0-9259-409a-bd22-3efb5c3fa5fa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.657940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 299/582] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:31.658755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30ff34-c24d-4bf0-ae87-9b2898a9487e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.685405 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:31.695335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276845d-b8b4-400b-a6e7-a6eec7863689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.696636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276845d-b8b4-400b-a6e7-a6eec7863689 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.696908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276845d-b8b4-400b-a6e7-a6eec7863689 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.697168 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276845d-b8b4-400b-a6e7-a6eec7863689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.707423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276845d-b8b4-400b-a6e7-a6eec7863689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.708894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276845d-b8b4-400b-a6e7-a6eec7863689 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0276845d-b8b4-400b-a6e7-a6eec7863689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:31.722360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:31.723472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:31.724597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:31.724597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:31.724597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:31.724597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 284/583] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:31.725210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276845d-b8b4-400b-a6e7-a6eec7863689 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.757249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30ff34-c24d-4bf0-ae87-9b2898a9487e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.784163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:31.791771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28ba5a-0bde-467b-a09e-fa88f9a3ee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.792381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28ba5a-0bde-467b-a09e-fa88f9a3ee38 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.792663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28ba5a-0bde-467b-a09e-fa88f9a3ee38 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.792928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28ba5a-0bde-467b-a09e-fa88f9a3ee38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.793631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28ba5a-0bde-467b-a09e-fa88f9a3ee38 None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=b8785c8c5ac84393afcaf88e891f3b7a) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:31.793934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28ba5a-0bde-467b-a09e-fa88f9a3ee38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.795676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28ba5a-0bde-467b-a09e-fa88f9a3ee38 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.800846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 300/584] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/role_assignments?user.id=b8785c8c5ac84393afcaf88e891f3b7a&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:31.801610 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28ba5a-0bde-467b-a09e-fa88f9a3ee38 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.846971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276845d-b8b4-400b-a6e7-a6eec7863689 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.885524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5574ba1-7ef6-4138-b307-ec49e3a956bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.886149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5574ba1-7ef6-4138-b307-ec49e3a956bf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.886425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5574ba1-7ef6-4138-b307-ec49e3a956bf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.886701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5574ba1-7ef6-4138-b307-ec49e3a956bf None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.887364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5574ba1-7ef6-4138-b307-ec49e3a956bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:31.887664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5574ba1-7ef6-4138-b307-ec49e3a956bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.889112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5574ba1-7ef6-4138-b307-ec49e3a956bf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.907702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 285/585] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:31.907702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5574ba1-7ef6-4138-b307-ec49e3a956bf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:31.908692 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:31.923179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28ba5a-0bde-467b-a09e-fa88f9a3ee38 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:31.968673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d3c6e-ce20-421f-942c-41bf90fbf1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:31.969578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d3c6e-ce20-421f-942c-41bf90fbf1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:31.969899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d3c6e-ce20-421f-942c-41bf90fbf1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:31.970112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d3c6e-ce20-421f-942c-41bf90fbf1f3 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:31.970857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d3c6e-ce20-421f-942c-41bf90fbf1f3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:31.971154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d3c6e-ce20-421f-942c-41bf90fbf1f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:31.972613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d3c6e-ce20-421f-942c-41bf90fbf1f3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:31.991542 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 301/586] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:31.992427 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d3c6e-ce20-421f-942c-41bf90fbf1f3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:32.017543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5574ba1-7ef6-4138-b307-ec49e3a956bf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:32.045277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:32.061240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43ecfd-c534-427e-9673-55129fd6bda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:32.062132 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43ecfd-c534-427e-9673-55129fd6bda4 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:32.062546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43ecfd-c534-427e-9673-55129fd6bda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:32.062961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43ecfd-c534-427e-9673-55129fd6bda4 None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/5952db29e6fc4ddbb9aeb0522497312c/roles/bd4e99b9d39b4ad2abf66671f7a35d61` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:32.080152 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43ecfd-c534-427e-9673-55129fd6bda4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=5952db29e6fc4ddbb9aeb0522497312c, role_id=bd4e99b9d39b4ad2abf66671f7a35d61)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:32.081837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43ecfd-c534-427e-9673-55129fd6bda4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:32.091938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d3c6e-ce20-421f-942c-41bf90fbf1f3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:32.104555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 286/587] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:16:31 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/5952db29e6fc4ddbb9aeb0522497312c/roles/bd4e99b9d39b4ad2abf66671f7a35d61 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:32.104555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43ecfd-c534-427e-9673-55129fd6bda4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:32.139312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e72f0-5262-4705-adb9-2a27c71fb52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:32.140688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e72f0-5262-4705-adb9-2a27c71fb52b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:32.141115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e72f0-5262-4705-adb9-2a27c71fb52b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:32.141437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e72f0-5262-4705-adb9-2a27c71fb52b None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:32.142107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e72f0-5262-4705-adb9-2a27c71fb52b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:32.142457 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e72f0-5262-4705-adb9-2a27c71fb52b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:32.145306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e72f0-5262-4705-adb9-2a27c71fb52b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:32.161438 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 302/588] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:32.163268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e72f0-5262-4705-adb9-2a27c71fb52b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:32.179645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:32.224864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43ecfd-c534-427e-9673-55129fd6bda4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:32.263957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760b27e-25cf-448b-a704-0b48797c9c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:32.264806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760b27e-25cf-448b-a704-0b48797c9c45 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:32.265145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760b27e-25cf-448b-a704-0b48797c9c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:32.265485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760b27e-25cf-448b-a704-0b48797c9c45 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:32.274530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760b27e-25cf-448b-a704-0b48797c9c45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:32.276029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760b27e-25cf-448b-a704-0b48797c9c45 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e760b27e-25cf-448b-a704-0b48797c9c45 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:32.291633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:32.292696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:32.293665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:32.293665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:32.293665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:32.293665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:32.293665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:32.293665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:32.293665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:32.293665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:32.293665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 30 19:16:32.293665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:32.296518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 287/589] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:32.298442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760b27e-25cf-448b-a704-0b48797c9c45 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:32.313786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e72f0-5262-4705-adb9-2a27c71fb52b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:32.339399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:32.355246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6c0fa-ab5d-454e-9a2d-e5b99b7d3a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:32.356455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c0fa-ab5d-454e-9a2d-e5b99b7d3a96 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:32.356698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c0fa-ab5d-454e-9a2d-e5b99b7d3a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:32.356967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c0fa-ab5d-454e-9a2d-e5b99b7d3a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:32.370008 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6c0fa-ab5d-454e-9a2d-e5b99b7d3a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:32.371451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6c0fa-ab5d-454e-9a2d-e5b99b7d3a96 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-23f6c0fa-ab5d-454e-9a2d-e5b99b7d3a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:32.387530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:32.389031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:32.389956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:32.389956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:32.389956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:32.389956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 303/590] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:32.389956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6c0fa-ab5d-454e-9a2d-e5b99b7d3a96 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:32.440890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760b27e-25cf-448b-a704-0b48797c9c45 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:32.458778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:32.481524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774f05b-b2c4-433f-b7c8-8635c216bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:32.482601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774f05b-b2c4-433f-b7c8-8635c216bc58 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:32.483079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774f05b-b2c4-433f-b7c8-8635c216bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:32.483544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774f05b-b2c4-433f-b7c8-8635c216bc58 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:32.492537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774f05b-b2c4-433f-b7c8-8635c216bc58 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:32.494414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774f05b-b2c4-433f-b7c8-8635c216bc58 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:32.504664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6c0fa-ab5d-454e-9a2d-e5b99b7d3a96 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9774f05b-b2c4-433f-b7c8-8635c216bc58 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:32.507332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:32.508318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:32.509350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:32.509350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:32.509350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:32.509350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:32.509350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:32.509350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:32.509350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:32.509350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:32.509350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 19:16:32.509350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:32.511975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 288/591] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:32.513148 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774f05b-b2c4-433f-b7c8-8635c216bc58 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:32.541411 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e59bc4-bdb7-42f2-90ae-95631ed6a53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:32.541951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e59bc4-bdb7-42f2-90ae-95631ed6a53b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:32.542260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e59bc4-bdb7-42f2-90ae-95631ed6a53b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:32.542662 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e59bc4-bdb7-42f2-90ae-95631ed6a53b None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:32.543407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e59bc4-bdb7-42f2-90ae-95631ed6a53b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:32.544040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e59bc4-bdb7-42f2-90ae-95631ed6a53b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:32.545767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e59bc4-bdb7-42f2-90ae-95631ed6a53b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:32.568442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 304/592] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 30 19:16:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:32.569682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e59bc4-bdb7-42f2-90ae-95631ed6a53b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:32.621232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:32.630537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774f05b-b2c4-433f-b7c8-8635c216bc58 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:32.669620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deed971-8d51-49c4-8ad0-8ffacb68b3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:32.670487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deed971-8d51-49c4-8ad0-8ffacb68b3eb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:32.670906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deed971-8d51-49c4-8ad0-8ffacb68b3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:32.671339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deed971-8d51-49c4-8ad0-8ffacb68b3eb None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:32.672136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deed971-8d51-49c4-8ad0-8ffacb68b3eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:32.672609 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deed971-8d51-49c4-8ad0-8ffacb68b3eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:32.674116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deed971-8d51-49c4-8ad0-8ffacb68b3eb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:32.692156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 289/593] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:32.693393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deed971-8d51-49c4-8ad0-8ffacb68b3eb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:32.704682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e59bc4-bdb7-42f2-90ae-95631ed6a53b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:32.727152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:32.749215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a55f6-e133-4958-a7e9-9e220016d4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:32.750470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a55f6-e133-4958-a7e9-9e220016d4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:32.750860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a55f6-e133-4958-a7e9-9e220016d4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:32.751235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a55f6-e133-4958-a7e9-9e220016d4f2 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:32.752328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a55f6-e133-4958-a7e9-9e220016d4f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:32.752773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a55f6-e133-4958-a7e9-9e220016d4f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:32.754562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a55f6-e133-4958-a7e9-9e220016d4f2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:32.774744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 305/594] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Sep 30 19:16:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:32.776049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a55f6-e133-4958-a7e9-9e220016d4f2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:32.800513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deed971-8d51-49c4-8ad0-8ffacb68b3eb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:32.836504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febe0b3-4bce-442a-9ecb-fd599612f634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:32.837360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe0b3-4bce-442a-9ecb-fd599612f634 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:32.837781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe0b3-4bce-442a-9ecb-fd599612f634 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:32.838245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe0b3-4bce-442a-9ecb-fd599612f634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:32.848914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febe0b3-4bce-442a-9ecb-fd599612f634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:32.850552 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febe0b3-4bce-442a-9ecb-fd599612f634 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4febe0b3-4bce-442a-9ecb-fd599612f634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:32.867618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:32.868848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:32.869947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:32.869947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:32.869947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:32.875904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 290/595] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:32.877003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febe0b3-4bce-442a-9ecb-fd599612f634 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:32.880979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:32.894292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a55f6-e133-4958-a7e9-9e220016d4f2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:32.935325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dedf1-b865-4097-a903-87a67b827e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:32.936597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dedf1-b865-4097-a903-87a67b827e06 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:32.937489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dedf1-b865-4097-a903-87a67b827e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:32.938219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dedf1-b865-4097-a903-87a67b827e06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:32.947008 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dedf1-b865-4097-a903-87a67b827e06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:32.948816 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dedf1-b865-4097-a903-87a67b827e06 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-170dedf1-b865-4097-a903-87a67b827e06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:32.962644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:32.963584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:32.969885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:32.969885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:32.969885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:32.969885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:32.969885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:32.969885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:32.969885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:32.969885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:32.970458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 306/596] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:32.973438 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dedf1-b865-4097-a903-87a67b827e06 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:32.993726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febe0b3-4bce-442a-9ecb-fd599612f634 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:33.016481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:33.033259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f73478-3aa0-4d10-96d6-98ba5f0e7c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:33.033946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f73478-3aa0-4d10-96d6-98ba5f0e7c82 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:33.034234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f73478-3aa0-4d10-96d6-98ba5f0e7c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:33.034507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f73478-3aa0-4d10-96d6-98ba5f0e7c82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:33.049844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f73478-3aa0-4d10-96d6-98ba5f0e7c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:33.052085 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f73478-3aa0-4d10-96d6-98ba5f0e7c82 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-38f73478-3aa0-4d10-96d6-98ba5f0e7c82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:33.074793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:33.076110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:33.077479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:33.077479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:33.077479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:33.077479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 291/597] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:33.078129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f73478-3aa0-4d10-96d6-98ba5f0e7c82 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:33.093818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dedf1-b865-4097-a903-87a67b827e06 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:33.132548 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff19d77-6f89-4716-ab60-88c786681e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:33.134115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff19d77-6f89-4716-ab60-88c786681e4e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:33.134456 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff19d77-6f89-4716-ab60-88c786681e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:33.134791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff19d77-6f89-4716-ab60-88c786681e4e None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:33.135811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff19d77-6f89-4716-ab60-88c786681e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:33.136321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff19d77-6f89-4716-ab60-88c786681e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:33.138006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff19d77-6f89-4716-ab60-88c786681e4e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:33.148771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:33.153298 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 307/598] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:33.157087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff19d77-6f89-4716-ab60-88c786681e4e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:33.214647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f73478-3aa0-4d10-96d6-98ba5f0e7c82 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:33.256553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9b9f8-fb8c-4408-88a3-bb6eec17f3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:33.257399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9b9f8-fb8c-4408-88a3-bb6eec17f3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:33.257822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9b9f8-fb8c-4408-88a3-bb6eec17f3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:33.258266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9b9f8-fb8c-4408-88a3-bb6eec17f3b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:33.259115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9b9f8-fb8c-4408-88a3-bb6eec17f3b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:33.259566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9b9f8-fb8c-4408-88a3-bb6eec17f3b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:33.261223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9b9f8-fb8c-4408-88a3-bb6eec17f3b7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:33.266651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:33.275063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 292/599] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:33.276308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9b9f8-fb8c-4408-88a3-bb6eec17f3b7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:33.282736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff19d77-6f89-4716-ab60-88c786681e4e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:33.330710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c6688-7a30-446c-bb95-b48a0b4cef27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:33.331810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6688-7a30-446c-bb95-b48a0b4cef27 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:33.332244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6688-7a30-446c-bb95-b48a0b4cef27 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:33.332705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c6688-7a30-446c-bb95-b48a0b4cef27 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:33.333889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c6688-7a30-446c-bb95-b48a0b4cef27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:33.334343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c6688-7a30-446c-bb95-b48a0b4cef27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:33.336904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c6688-7a30-446c-bb95-b48a0b4cef27 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:33.353696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 308/600] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:33.354762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c6688-7a30-446c-bb95-b48a0b4cef27 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:33.392582 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9b9f8-fb8c-4408-88a3-bb6eec17f3b7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:33.420751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:33.431351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775d2a6-d0b4-4284-886f-12f3f88f2fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:33.432145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775d2a6-d0b4-4284-886f-12f3f88f2fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:33.432567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775d2a6-d0b4-4284-886f-12f3f88f2fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:33.432982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775d2a6-d0b4-4284-886f-12f3f88f2fc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:33.439988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775d2a6-d0b4-4284-886f-12f3f88f2fc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:33.441584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775d2a6-d0b4-4284-886f-12f3f88f2fc2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3775d2a6-d0b4-4284-886f-12f3f88f2fc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:33.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:33.454799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:33.455926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:33.455926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:33.455926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:33.455926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:33.455926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:33.455926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:33.455926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:33.455926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:33.457493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 293/601] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:33.458134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775d2a6-d0b4-4284-886f-12f3f88f2fc2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:33.494124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c6688-7a30-446c-bb95-b48a0b4cef27 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:33.538916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f855b9b-ad7a-4b69-9caa-5f04f5a4684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:33.540120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f855b9b-ad7a-4b69-9caa-5f04f5a4684b None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:33.540576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f855b9b-ad7a-4b69-9caa-5f04f5a4684b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:33.541013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f855b9b-ad7a-4b69-9caa-5f04f5a4684b None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/65d3020e1f6c414caabbaa185fb7ed48/roles/bd4e99b9d39b4ad2abf66671f7a35d61` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:33.557010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f855b9b-ad7a-4b69-9caa-5f04f5a4684b None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=65d3020e1f6c414caabbaa185fb7ed48, role_id=bd4e99b9d39b4ad2abf66671f7a35d61)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:33.558573 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f855b9b-ad7a-4b69-9caa-5f04f5a4684b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:33.564598 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775d2a6-d0b4-4284-886f-12f3f88f2fc2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:33.580607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 309/602] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:16:33 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/65d3020e1f6c414caabbaa185fb7ed48/roles/bd4e99b9d39b4ad2abf66671f7a35d61 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:33.588482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f855b9b-ad7a-4b69-9caa-5f04f5a4684b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:33.607201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:33.641500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f2f9a-f718-4092-b729-68da14e026c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:33.642310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f2f9a-f718-4092-b729-68da14e026c5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:33.643130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f2f9a-f718-4092-b729-68da14e026c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:33.643520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f2f9a-f718-4092-b729-68da14e026c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:33.661001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f2f9a-f718-4092-b729-68da14e026c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:33.664703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f2f9a-f718-4092-b729-68da14e026c5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f57f2f9a-f718-4092-b729-68da14e026c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:33.696243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:33.697429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:33.698558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:33.698558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:33.698558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:33.698558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:33.698558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:33.698558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:33.698558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:33.698558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:33.702298 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 294/603] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:33.711896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f2f9a-f718-4092-b729-68da14e026c5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:33.782416 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:33.802447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f855b9b-ad7a-4b69-9caa-5f04f5a4684b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:33.843399 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2a894-860a-4b49-b364-58acd6df24ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:33.843946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2a894-860a-4b49-b364-58acd6df24ee None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:33.844222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2a894-860a-4b49-b364-58acd6df24ee None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:33.844526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2a894-860a-4b49-b364-58acd6df24ee None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:33.845664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2a894-860a-4b49-b364-58acd6df24ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:33.845664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2a894-860a-4b49-b364-58acd6df24ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:33.847236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2a894-860a-4b49-b364-58acd6df24ee None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:33.861866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f2f9a-f718-4092-b729-68da14e026c5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:33.865310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 310/604] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:33.871874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2a894-860a-4b49-b364-58acd6df24ee None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:33.904544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dc0b5-f032-40b4-a8e6-8cb29a0c5db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:33.905990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dc0b5-f032-40b4-a8e6-8cb29a0c5db0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:33.906535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dc0b5-f032-40b4-a8e6-8cb29a0c5db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:33.907002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dc0b5-f032-40b4-a8e6-8cb29a0c5db0 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:33.908534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972dc0b5-f032-40b4-a8e6-8cb29a0c5db0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:33.909012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972dc0b5-f032-40b4-a8e6-8cb29a0c5db0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:33.911170 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972dc0b5-f032-40b4-a8e6-8cb29a0c5db0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:33.928248 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 295/605] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:33.934591 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dc0b5-f032-40b4-a8e6-8cb29a0c5db0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:33.987019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:34.028282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2a894-860a-4b49-b364-58acd6df24ee None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:34.067202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d59a7-096e-4928-b074-f93f637a3877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:34.067871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d59a7-096e-4928-b074-f93f637a3877 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:34.068149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d59a7-096e-4928-b074-f93f637a3877 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:34.068466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d59a7-096e-4928-b074-f93f637a3877 None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/58a9b371434243daa4176215daefb79c/roles/bd4e99b9d39b4ad2abf66671f7a35d61` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:34.078443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:34.087135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d59a7-096e-4928-b074-f93f637a3877 None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=58a9b371434243daa4176215daefb79c, role_id=bd4e99b9d39b4ad2abf66671f7a35d61)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:34.087762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dc0b5-f032-40b4-a8e6-8cb29a0c5db0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:34.088599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d59a7-096e-4928-b074-f93f637a3877 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:34.105731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 311/606] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:16:33 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/58a9b371434243daa4176215daefb79c/roles/bd4e99b9d39b4ad2abf66671f7a35d61 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:34.108942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dcef3-bf64-4fea-baa5-6a9bc3977b8d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:34.109399 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dcef3-bf64-4fea-baa5-6a9bc3977b8d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:34.109833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dcef3-bf64-4fea-baa5-6a9bc3977b8d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:34.117372 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 312/607] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:34.122385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aceb17-77e8-4b91-92d0-1fdb72ce334b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:34.123065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aceb17-77e8-4b91-92d0-1fdb72ce334b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:34.123626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aceb17-77e8-4b91-92d0-1fdb72ce334b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:34.157221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b418ab-d3f5-4b39-baca-9738b7ff1d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:34.158173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b418ab-d3f5-4b39-baca-9738b7ff1d08 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:34.160899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b418ab-d3f5-4b39-baca-9738b7ff1d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:34.161372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b418ab-d3f5-4b39-baca-9738b7ff1d08 None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/18ea49d1f6c040f18511535b7e51a53b/roles/bd4e99b9d39b4ad2abf66671f7a35d61` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:34.164955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-28aceb17-77e8-4b91-92d0-1fdb72ce334b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:34.166550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-28aceb17-77e8-4b91-92d0-1fdb72ce334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:34.193821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b418ab-d3f5-4b39-baca-9738b7ff1d08 None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=18ea49d1f6c040f18511535b7e51a53b, role_id=bd4e99b9d39b4ad2abf66671f7a35d61)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:34.195484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b418ab-d3f5-4b39-baca-9738b7ff1d08 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:34.243529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 296/608] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:16:33 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/18ea49d1f6c040f18511535b7e51a53b/roles/bd4e99b9d39b4ad2abf66671f7a35d61 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:34.252019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 313/609] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:34.446255 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d372-6c27-4abc-8a1e-c8d6846fc418 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:34.446723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d372-6c27-4abc-8a1e-c8d6846fc418 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:34.447108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d372-6c27-4abc-8a1e-c8d6846fc418 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:34.448903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 297/610] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:34.454570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ffb59-677c-491d-ae2f-eb743d57d16f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:34.454993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ffb59-677c-491d-ae2f-eb743d57d16f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:34.455371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ffb59-677c-491d-ae2f-eb743d57d16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:34.474805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-310ffb59-677c-491d-ae2f-eb743d57d16f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:34.476116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-310ffb59-677c-491d-ae2f-eb743d57d16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:34.524437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:34.551398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 314/611] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:34.569945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997d372-6c27-4abc-8a1e-c8d6846fc418 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:34.700715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997d372-6c27-4abc-8a1e-c8d6846fc418 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:34.735637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71948e5-8c3d-46c5-9edb-0953206ac21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:34.736577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71948e5-8c3d-46c5-9edb-0953206ac21c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:34.736993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71948e5-8c3d-46c5-9edb-0953206ac21c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:34.737430 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71948e5-8c3d-46c5-9edb-0953206ac21c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:34.748479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71948e5-8c3d-46c5-9edb-0953206ac21c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e71948e5-8c3d-46c5-9edb-0953206ac21c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:34.753422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:34.754555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:34.755590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:34.762880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 298/612] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:34.764649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ffb59-677c-491d-ae2f-eb743d57d16f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:34.869054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ffb59-677c-491d-ae2f-eb743d57d16f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:34.878981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71948e5-8c3d-46c5-9edb-0953206ac21c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:34.891608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:34.934435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415d3ac-78ba-42ca-931e-d02c176b3153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:34.937083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415d3ac-78ba-42ca-931e-d02c176b3153 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:34.939205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415d3ac-78ba-42ca-931e-d02c176b3153 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:34.939631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415d3ac-78ba-42ca-931e-d02c176b3153 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:34.940461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415d3ac-78ba-42ca-931e-d02c176b3153 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:34.942686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415d3ac-78ba-42ca-931e-d02c176b3153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:34.944212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415d3ac-78ba-42ca-931e-d02c176b3153 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:34.959802 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 315/613] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:34.967155 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415d3ac-78ba-42ca-931e-d02c176b3153 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:35.031285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:35.051530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71948e5-8c3d-46c5-9edb-0953206ac21c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:35.118996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3c416-aa10-40f7-8d02-1a8873d8ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:35.119627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3c416-aa10-40f7-8d02-1a8873d8ae14 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:35.119905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3c416-aa10-40f7-8d02-1a8873d8ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:35.120170 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3c416-aa10-40f7-8d02-1a8873d8ae14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:35.140763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3c416-aa10-40f7-8d02-1a8873d8ae14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6db3c416-aa10-40f7-8d02-1a8873d8ae14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:35.145594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:35.146518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:35.148176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 299/614] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:35.152435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415d3ac-78ba-42ca-931e-d02c176b3153 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:35.154638 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3c416-aa10-40f7-8d02-1a8873d8ae14 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:35.207916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8656812-d9ee-4871-8c91-000bb298a5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:35.208969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8656812-d9ee-4871-8c91-000bb298a5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:35.210422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8656812-d9ee-4871-8c91-000bb298a5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:35.212032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8656812-d9ee-4871-8c91-000bb298a5a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:35.238771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8656812-d9ee-4871-8c91-000bb298a5a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:35.240586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8656812-d9ee-4871-8c91-000bb298a5a1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a8656812-d9ee-4871-8c91-000bb298a5a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:35.268322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:35.269387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:35.270300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:35.270300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:35.270300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:35.280596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 316/615] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:35.285936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8656812-d9ee-4871-8c91-000bb298a5a1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:35.308044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:35.408505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8656812-d9ee-4871-8c91-000bb298a5a1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:35.420848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:35.438250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3c416-aa10-40f7-8d02-1a8873d8ae14 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:35.446528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d51646-c9a5-4698-9a27-89125daa2350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:35.447426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d51646-c9a5-4698-9a27-89125daa2350 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:35.447904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d51646-c9a5-4698-9a27-89125daa2350 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:35.448398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d51646-c9a5-4698-9a27-89125daa2350 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:35.457013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d51646-c9a5-4698-9a27-89125daa2350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:35.457013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d51646-c9a5-4698-9a27-89125daa2350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:35.457013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d51646-c9a5-4698-9a27-89125daa2350 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:35.470929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 317/616] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:35.477249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d51646-c9a5-4698-9a27-89125daa2350 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:35.586664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d51646-c9a5-4698-9a27-89125daa2350 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:35.589801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abb091-f419-4f8e-83bf-3657f8bd47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:35.590278 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb091-f419-4f8e-83bf-3657f8bd47fe None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:35.590563 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb091-f419-4f8e-83bf-3657f8bd47fe None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:35.590834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb091-f419-4f8e-83bf-3657f8bd47fe None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:35.591504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abb091-f419-4f8e-83bf-3657f8bd47fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:35.591804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abb091-f419-4f8e-83bf-3657f8bd47fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:35.593222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abb091-f419-4f8e-83bf-3657f8bd47fe None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:35.626820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 300/617] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:35.629356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abb091-f419-4f8e-83bf-3657f8bd47fe None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:35.631647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63853f-3bcc-44c2-9bba-90cdf216d4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:35.632488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63853f-3bcc-44c2-9bba-90cdf216d4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:35.634635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63853f-3bcc-44c2-9bba-90cdf216d4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:35.635074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63853f-3bcc-44c2-9bba-90cdf216d4e5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:35.648941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63853f-3bcc-44c2-9bba-90cdf216d4e5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:35.652012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63853f-3bcc-44c2-9bba-90cdf216d4e5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-4f63853f-3bcc-44c2-9bba-90cdf216d4e5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:35.672483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:35.673714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:35.674896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:35.674896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:35.674896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:35.674896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:35.674896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:35.674896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:35.674896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:35.674896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:35.674896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:35.674896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:35.682441 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 318/618] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:35.689069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63853f-3bcc-44c2-9bba-90cdf216d4e5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:35.724845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:35.807977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63853f-3bcc-44c2-9bba-90cdf216d4e5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:35.843012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4e1e7-571e-4215-9d27-eaf1fbeb00e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:35.843900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4e1e7-571e-4215-9d27-eaf1fbeb00e6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:35.844317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4e1e7-571e-4215-9d27-eaf1fbeb00e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:35.844893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4e1e7-571e-4215-9d27-eaf1fbeb00e6 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:35.845749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4e1e7-571e-4215-9d27-eaf1fbeb00e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:35.846181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4e1e7-571e-4215-9d27-eaf1fbeb00e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:35.847762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4e1e7-571e-4215-9d27-eaf1fbeb00e6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:35.861396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abb091-f419-4f8e-83bf-3657f8bd47fe None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:35.871164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 319/619] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 30 19:16:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:35.876857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4e1e7-571e-4215-9d27-eaf1fbeb00e6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:35.907216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:35.917485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defaa4c-d493-43a3-8b06-a1dfb3349fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:35.918117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defaa4c-d493-43a3-8b06-a1dfb3349fbe None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:35.918391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defaa4c-d493-43a3-8b06-a1dfb3349fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:35.918658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defaa4c-d493-43a3-8b06-a1dfb3349fbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:35.934509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defaa4c-d493-43a3-8b06-a1dfb3349fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:35.935969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defaa4c-d493-43a3-8b06-a1dfb3349fbe None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7defaa4c-d493-43a3-8b06-a1dfb3349fbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:35.962441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:35.963478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:35.964473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:35.964473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:35.964473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:35.964654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 301/620] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:35.972981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defaa4c-d493-43a3-8b06-a1dfb3349fbe None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:36.008567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:36.073483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4e1e7-571e-4215-9d27-eaf1fbeb00e6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:36.144257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc1635-3a00-40d7-a456-41ab8759737d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:36.144928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc1635-3a00-40d7-a456-41ab8759737d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:36.145202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc1635-3a00-40d7-a456-41ab8759737d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:36.145469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc1635-3a00-40d7-a456-41ab8759737d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:36.160977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc1635-3a00-40d7-a456-41ab8759737d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:36.162688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc1635-3a00-40d7-a456-41ab8759737d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:36.173409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defaa4c-d493-43a3-8b06-a1dfb3349fbe None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-7ffc1635-3a00-40d7-a456-41ab8759737d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:36.190465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:36.192305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:36.194100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:36.194100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:36.194100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:36.194100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 320/621] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:36.201269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc1635-3a00-40d7-a456-41ab8759737d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:36.236153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5dbf5-2196-4bef-bdc6-59007cabf6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:36.236817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5dbf5-2196-4bef-bdc6-59007cabf6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:36.237097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5dbf5-2196-4bef-bdc6-59007cabf6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:36.237359 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5dbf5-2196-4bef-bdc6-59007cabf6b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:36.238007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5dbf5-2196-4bef-bdc6-59007cabf6b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:36.238396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5dbf5-2196-4bef-bdc6-59007cabf6b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:36.239865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5dbf5-2196-4bef-bdc6-59007cabf6b3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:36.253592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 302/622] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:36.254920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fc631-bc64-490b-a23f-ca0dbf65f103 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:36.255212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fc631-bc64-490b-a23f-ca0dbf65f103 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:36.255482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fc631-bc64-490b-a23f-ca0dbf65f103 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:36.257008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 303/623] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:36.260885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fc631-bc64-490b-a23f-ca0dbf65f103 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:36.307651 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:36.334419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc1635-3a00-40d7-a456-41ab8759737d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:36.357804 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:36.373298 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dee78-96ba-4203-8d87-57813a1c25d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:36.374155 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dee78-96ba-4203-8d87-57813a1c25d8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:36.374542 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dee78-96ba-4203-8d87-57813a1c25d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:36.374920 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dee78-96ba-4203-8d87-57813a1c25d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:36.375766 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47dee78-96ba-4203-8d87-57813a1c25d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:36.376218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47dee78-96ba-4203-8d87-57813a1c25d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:36.377890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47dee78-96ba-4203-8d87-57813a1c25d8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:36.379369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fc631-bc64-490b-a23f-ca0dbf65f103 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:36.398941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 321/624] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:36.400538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e729b-df82-48bd-b73d-97875cbf8b63 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:36.400932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e729b-df82-48bd-b73d-97875cbf8b63 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:36.401306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e729b-df82-48bd-b73d-97875cbf8b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:36.421306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f431be-2441-480c-99c2-0f48f586dcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:36.422176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f431be-2441-480c-99c2-0f48f586dcde None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:36.422607 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f431be-2441-480c-99c2-0f48f586dcde None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:36.423038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f431be-2441-480c-99c2-0f48f586dcde None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:36.423476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-630e729b-df82-48bd-b73d-97875cbf8b63 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:36.424969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-630e729b-df82-48bd-b73d-97875cbf8b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:36.431947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f431be-2441-480c-99c2-0f48f586dcde None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:36.433655 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f431be-2441-480c-99c2-0f48f586dcde None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-32f431be-2441-480c-99c2-0f48f586dcde None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:36.448047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:36.449302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:36.450484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:36.450484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:36.450484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:36.450484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:36.450484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:36.450484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:36.450484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:36.450484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:36.450484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:36.450484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:36.452800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 304/625] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 30 19:16:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:36.453818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f431be-2441-480c-99c2-0f48f586dcde None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:36.512587 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 322/626] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:36.513670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e729b-df82-48bd-b73d-97875cbf8b63 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:36.641706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:36.650579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f431be-2441-480c-99c2-0f48f586dcde None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:36.652259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e729b-df82-48bd-b73d-97875cbf8b63 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:36.665507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:36.692356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53014a-19af-49e6-ba5c-5a19e057197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:36.693402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53014a-19af-49e6-ba5c-5a19e057197b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:36.693819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53014a-19af-49e6-ba5c-5a19e057197b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:36.694229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53014a-19af-49e6-ba5c-5a19e057197b None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:36.695188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53014a-19af-49e6-ba5c-5a19e057197b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:36.696206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53014a-19af-49e6-ba5c-5a19e057197b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:36.698389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53014a-19af-49e6-ba5c-5a19e057197b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:36.719433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 323/627] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 30 19:16:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:36.725039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53014a-19af-49e6-ba5c-5a19e057197b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:36.738778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10175afc-34a9-42a6-a139-c2ea73b1f831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:36.739271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10175afc-34a9-42a6-a139-c2ea73b1f831 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:36.739656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10175afc-34a9-42a6-a139-c2ea73b1f831 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:36.740032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10175afc-34a9-42a6-a139-c2ea73b1f831 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:36.761285 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10175afc-34a9-42a6-a139-c2ea73b1f831 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:36.762842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10175afc-34a9-42a6-a139-c2ea73b1f831 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-10175afc-34a9-42a6-a139-c2ea73b1f831 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:36.801608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:36.802785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:36.804115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:36.804115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:36.804115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:36.804115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:36.804115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:36.804115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:36.804115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:36.804115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:36.804115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 305/628] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:36.809974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10175afc-34a9-42a6-a139-c2ea73b1f831 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:36.844415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53014a-19af-49e6-ba5c-5a19e057197b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:36.880817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e3ee8-4be3-4a18-aad8-3116daac9119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:36.881437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e3ee8-4be3-4a18-aad8-3116daac9119 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:36.881700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e3ee8-4be3-4a18-aad8-3116daac9119 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:36.882157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e3ee8-4be3-4a18-aad8-3116daac9119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:36.917204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:36.921849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10175afc-34a9-42a6-a139-c2ea73b1f831 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:36.949547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e3ee8-4be3-4a18-aad8-3116daac9119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:36.950988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e3ee8-4be3-4a18-aad8-3116daac9119 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:36.954547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c296b0-d20d-491d-bebe-4de37244c5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:36.955142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c296b0-d20d-491d-bebe-4de37244c5aa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:36.955420 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c296b0-d20d-491d-bebe-4de37244c5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:36.955689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c296b0-d20d-491d-bebe-4de37244c5aa None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:36.956392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c296b0-d20d-491d-bebe-4de37244c5aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:36.956706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c296b0-d20d-491d-bebe-4de37244c5aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:36.958123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c296b0-d20d-491d-bebe-4de37244c5aa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:36.970312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 306/629] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:36.971431 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40ed8-726a-4110-909e-2982abcd6675 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:36.971721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40ed8-726a-4110-909e-2982abcd6675 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:36.971996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40ed8-726a-4110-909e-2982abcd6675 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:36.973214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 307/630] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:36.973909 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb40ed8-726a-4110-909e-2982abcd6675 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:36.995463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:37.053714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 324/631] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:37.055375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4cd47-0c8e-4a04-88c7-83025028018b None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:37.055682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4cd47-0c8e-4a04-88c7-83025028018b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:37.055992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4cd47-0c8e-4a04-88c7-83025028018b None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:37.057420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 325/632] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:37.062176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4cd47-0c8e-4a04-88c7-83025028018b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:37.079766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb40ed8-726a-4110-909e-2982abcd6675 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:37.112616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d400a05-9acf-406d-8fdd-83797ee5c736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:37.113229 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d400a05-9acf-406d-8fdd-83797ee5c736 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:37.113502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d400a05-9acf-406d-8fdd-83797ee5c736 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:37.113774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d400a05-9acf-406d-8fdd-83797ee5c736 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:37.123467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d400a05-9acf-406d-8fdd-83797ee5c736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:37.127148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5d400a05-9acf-406d-8fdd-83797ee5c736 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:37.128161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:37.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:37.130157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 308/633] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:37.132084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911051a2-caae-42c1-b9d0-0cecef08544d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:37.132084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911051a2-caae-42c1-b9d0-0cecef08544d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:37.132084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911051a2-caae-42c1-b9d0-0cecef08544d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:37.149161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-911051a2-caae-42c1-b9d0-0cecef08544d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:37.150331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-911051a2-caae-42c1-b9d0-0cecef08544d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:37.167100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4cd47-0c8e-4a04-88c7-83025028018b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:37.199998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc765aa-8759-4604-b6b0-a8e591cfdcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:37.200682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc765aa-8759-4604-b6b0-a8e591cfdcab None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:37.200961 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc765aa-8759-4604-b6b0-a8e591cfdcab None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:37.201234 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc765aa-8759-4604-b6b0-a8e591cfdcab None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/b8785c8c5ac84393afcaf88e891f3b7a/roles/bd4e99b9d39b4ad2abf66671f7a35d61` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:37.214667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 309/634] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:37.214815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc765aa-8759-4604-b6b0-a8e591cfdcab None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=b8785c8c5ac84393afcaf88e891f3b7a, role_id=bd4e99b9d39b4ad2abf66671f7a35d61)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:37.215483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911051a2-caae-42c1-b9d0-0cecef08544d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:37.216187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc765aa-8759-4604-b6b0-a8e591cfdcab None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:37.229628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 326/635] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:16:37 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/b8785c8c5ac84393afcaf88e891f3b7a/roles/bd4e99b9d39b4ad2abf66671f7a35d61 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:37.230806 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c1007-9973-4f85-bd94-40b778504cd4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:37.231086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c1007-9973-4f85-bd94-40b778504cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:37.231349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c1007-9973-4f85-bd94-40b778504cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:37.247124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:37.254314 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c60c1007-9973-4f85-bd94-40b778504cd4 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:37.255651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c60c1007-9973-4f85-bd94-40b778504cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:37.342389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 327/636] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:37.343343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c1007-9973-4f85-bd94-40b778504cd4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:37.362269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911051a2-caae-42c1-b9d0-0cecef08544d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:37.417668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069eb3d-d7f2-44fb-bc8a-e4ad7a7feed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:37.418608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069eb3d-d7f2-44fb-bc8a-e4ad7a7feed3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:37.419336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:37.419665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069eb3d-d7f2-44fb-bc8a-e4ad7a7feed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:37.420608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069eb3d-d7f2-44fb-bc8a-e4ad7a7feed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:37.434988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069eb3d-d7f2-44fb-bc8a-e4ad7a7feed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:37.437134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069eb3d-d7f2-44fb-bc8a-e4ad7a7feed3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5069eb3d-d7f2-44fb-bc8a-e4ad7a7feed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:37.452583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:37.453578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:37.454504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:37.454504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:37.454504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:37.457431 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 310/637] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:37.462619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069eb3d-d7f2-44fb-bc8a-e4ad7a7feed3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:37.474744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c1007-9973-4f85-bd94-40b778504cd4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:37.511583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323c026-e8c7-422c-956b-24a75f72647a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:37.513951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323c026-e8c7-422c-956b-24a75f72647a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:37.514243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323c026-e8c7-422c-956b-24a75f72647a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:37.515761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323c026-e8c7-422c-956b-24a75f72647a None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:37.516574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323c026-e8c7-422c-956b-24a75f72647a None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:37.517000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323c026-e8c7-422c-956b-24a75f72647a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:37.518756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323c026-e8c7-422c-956b-24a75f72647a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:37.528137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 328/638] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:37.529362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323c026-e8c7-422c-956b-24a75f72647a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:37.536866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:37.587713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069eb3d-d7f2-44fb-bc8a-e4ad7a7feed3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:37.634818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90599600-4088-4360-b5e9-a7bbc51f2ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:37.635658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90599600-4088-4360-b5e9-a7bbc51f2ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:37.636105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90599600-4088-4360-b5e9-a7bbc51f2ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:37.636567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90599600-4088-4360-b5e9-a7bbc51f2ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:37.637928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90599600-4088-4360-b5e9-a7bbc51f2ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:37.638383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90599600-4088-4360-b5e9-a7bbc51f2ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:37.640623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90599600-4088-4360-b5e9-a7bbc51f2ce3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:37.650904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323c026-e8c7-422c-956b-24a75f72647a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:37.658194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 311/639] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:37.659281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90599600-4088-4360-b5e9-a7bbc51f2ce3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:37.673477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:37.700604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07f468-becd-4471-9801-e0431ea332c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:37.701235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07f468-becd-4471-9801-e0431ea332c1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:37.701509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07f468-becd-4471-9801-e0431ea332c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:37.701778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07f468-becd-4471-9801-e0431ea332c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:37.713394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07f468-becd-4471-9801-e0431ea332c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-af07f468-becd-4471-9801-e0431ea332c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:37.718656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:37.719807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:37.720996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:37.721743 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 329/640] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:37.722398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07f468-becd-4471-9801-e0431ea332c1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:37.780890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90599600-4088-4360-b5e9-a7bbc51f2ce3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:37.804546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:37.817847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b4825-9485-41a1-95e4-defd1138dc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:37.818624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b4825-9485-41a1-95e4-defd1138dc05 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:37.818999 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b4825-9485-41a1-95e4-defd1138dc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:37.819342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b4825-9485-41a1-95e4-defd1138dc05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:37.830568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b4825-9485-41a1-95e4-defd1138dc05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:37.832078 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b4825-9485-41a1-95e4-defd1138dc05 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:37.835087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07f468-becd-4471-9801-e0431ea332c1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7d7b4825-9485-41a1-95e4-defd1138dc05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:37.846896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:37.848072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:37.849204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:37.849204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:37.849204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:37.854803 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 312/641] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:37.854803 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b4825-9485-41a1-95e4-defd1138dc05 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:37.878698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f09ff-b7ed-4012-8bec-f41a557b03a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:37.879326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f09ff-b7ed-4012-8bec-f41a557b03a6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:37.879662 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f09ff-b7ed-4012-8bec-f41a557b03a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:37.879990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f09ff-b7ed-4012-8bec-f41a557b03a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:37.887409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f09ff-b7ed-4012-8bec-f41a557b03a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:37.888887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f09ff-b7ed-4012-8bec-f41a557b03a6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-933f09ff-b7ed-4012-8bec-f41a557b03a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:37.900973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:37.902140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:37.903083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:37.903083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:37.903083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:37.903083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:37.903083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:37.903083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:37.903083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:37.903083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:37.903083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 330/642] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:37.903597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f09ff-b7ed-4012-8bec-f41a557b03a6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:37.935882 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:37.965535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b4825-9485-41a1-95e4-defd1138dc05 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:37.999769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd908b73-35bd-44aa-ab2c-363fb5dc6ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.000424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd908b73-35bd-44aa-ab2c-363fb5dc6ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.000709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd908b73-35bd-44aa-ab2c-363fb5dc6ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.000982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd908b73-35bd-44aa-ab2c-363fb5dc6ed5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.011386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f09ff-b7ed-4012-8bec-f41a557b03a6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:38.011851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd908b73-35bd-44aa-ab2c-363fb5dc6ed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fd908b73-35bd-44aa-ab2c-363fb5dc6ed5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:38.017119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:38.018092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 313/643] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:38.019006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd908b73-35bd-44aa-ab2c-363fb5dc6ed5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:38.057012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fd33b-c6df-4557-8a30-bc96a1385009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.057339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:38.057998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fd33b-c6df-4557-8a30-bc96a1385009 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.058275 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fd33b-c6df-4557-8a30-bc96a1385009 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.058538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fd33b-c6df-4557-8a30-bc96a1385009 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.059197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fd33b-c6df-4557-8a30-bc96a1385009 None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:38.059496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fd33b-c6df-4557-8a30-bc96a1385009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:38.061057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fd33b-c6df-4557-8a30-bc96a1385009 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:38.069912 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 331/644] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:38.071144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fd33b-c6df-4557-8a30-bc96a1385009 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:38.129827 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd908b73-35bd-44aa-ab2c-363fb5dc6ed5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:38.164339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a617bff-dbe2-49be-afbb-9300429db573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.164917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a617bff-dbe2-49be-afbb-9300429db573 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.165251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a617bff-dbe2-49be-afbb-9300429db573 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.165524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a617bff-dbe2-49be-afbb-9300429db573 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.166213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a617bff-dbe2-49be-afbb-9300429db573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:38.166517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a617bff-dbe2-49be-afbb-9300429db573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:38.167891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a617bff-dbe2-49be-afbb-9300429db573 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:38.174642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:38.184034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 314/645] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:38.184034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a617bff-dbe2-49be-afbb-9300429db573 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:38.188294 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fd33b-c6df-4557-8a30-bc96a1385009 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:38.226227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5554d-ab7d-412d-abda-36d989b7d628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.226874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5554d-ab7d-412d-abda-36d989b7d628 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.227148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5554d-ab7d-412d-abda-36d989b7d628 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.227421 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5554d-ab7d-412d-abda-36d989b7d628 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.228111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5554d-ab7d-412d-abda-36d989b7d628 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:38.228436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5554d-ab7d-412d-abda-36d989b7d628 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:38.229856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5554d-ab7d-412d-abda-36d989b7d628 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:38.247649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 332/646] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:38.247649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5554d-ab7d-412d-abda-36d989b7d628 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:38.284001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:38.291652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a617bff-dbe2-49be-afbb-9300429db573 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:38.326770 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808a023-591c-4b1b-b13d-f01ec53c56e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.327721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808a023-591c-4b1b-b13d-f01ec53c56e8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.328008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808a023-591c-4b1b-b13d-f01ec53c56e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.328283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808a023-591c-4b1b-b13d-f01ec53c56e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.329283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808a023-591c-4b1b-b13d-f01ec53c56e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:38.329768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808a023-591c-4b1b-b13d-f01ec53c56e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:38.331107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808a023-591c-4b1b-b13d-f01ec53c56e8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:38.344494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 315/647] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:38.344494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808a023-591c-4b1b-b13d-f01ec53c56e8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:38.352729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5554d-ab7d-412d-abda-36d989b7d628 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:38.391145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff18de6-0a7b-4733-ae0e-b05f087a494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.391775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff18de6-0a7b-4733-ae0e-b05f087a494a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.392050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff18de6-0a7b-4733-ae0e-b05f087a494a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.392317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff18de6-0a7b-4733-ae0e-b05f087a494a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.403441 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff18de6-0a7b-4733-ae0e-b05f087a494a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:38.404915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff18de6-0a7b-4733-ae0e-b05f087a494a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-6ff18de6-0a7b-4733-ae0e-b05f087a494a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:38.419973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:38.421129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:38.422036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:38.422036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:38.422036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:38.422036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 333/648] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:38.422274 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff18de6-0a7b-4733-ae0e-b05f087a494a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:38.427920 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:38.458634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808a023-591c-4b1b-b13d-f01ec53c56e8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:38.492820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44f238-8e25-409a-8064-a7b2daa664af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.493441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44f238-8e25-409a-8064-a7b2daa664af None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.493715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44f238-8e25-409a-8064-a7b2daa664af None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.493979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44f238-8e25-409a-8064-a7b2daa664af None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.501386 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44f238-8e25-409a-8064-a7b2daa664af None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:38.502808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44f238-8e25-409a-8064-a7b2daa664af None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1f44f238-8e25-409a-8064-a7b2daa664af None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:38.514246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:38.519768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 316/649] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:38.521049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44f238-8e25-409a-8064-a7b2daa664af None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:38.538902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:38.539533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff18de6-0a7b-4733-ae0e-b05f087a494a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:38.576451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245c7b0-f8b3-4179-ad15-95faed9fa46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.577060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245c7b0-f8b3-4179-ad15-95faed9fa46d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.577339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245c7b0-f8b3-4179-ad15-95faed9fa46d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.577612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245c7b0-f8b3-4179-ad15-95faed9fa46d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.578332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245c7b0-f8b3-4179-ad15-95faed9fa46d None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=5952db29e6fc4ddbb9aeb0522497312c) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:38.578636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245c7b0-f8b3-4179-ad15-95faed9fa46d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:38.580507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245c7b0-f8b3-4179-ad15-95faed9fa46d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:38.586106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 334/650] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/role_assignments?user.id=5952db29e6fc4ddbb9aeb0522497312c&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:38.586874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245c7b0-f8b3-4179-ad15-95faed9fa46d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:38.647503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44f238-8e25-409a-8064-a7b2daa664af None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:38.678819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:38.698920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fea09-7d13-46ce-aeb2-166ed7c00ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.700288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fea09-7d13-46ce-aeb2-166ed7c00ced None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.700758 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fea09-7d13-46ce-aeb2-166ed7c00ced None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.701390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fea09-7d13-46ce-aeb2-166ed7c00ced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.717472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fea09-7d13-46ce-aeb2-166ed7c00ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:38.719888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fea09-7d13-46ce-aeb2-166ed7c00ced None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:38.720697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245c7b0-f8b3-4179-ad15-95faed9fa46d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d03fea09-7d13-46ce-aeb2-166ed7c00ced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:38.741331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:38.742627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:38.744074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:38.744074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:38.744074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:38.750593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 317/651] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:38.750593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fea09-7d13-46ce-aeb2-166ed7c00ced None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:38.773122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807a159-ec5f-4cef-a3f5-4529d6181b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.774104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807a159-ec5f-4cef-a3f5-4529d6181b77 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.774538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807a159-ec5f-4cef-a3f5-4529d6181b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.775563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807a159-ec5f-4cef-a3f5-4529d6181b77 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.776521 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807a159-ec5f-4cef-a3f5-4529d6181b77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:38.777056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807a159-ec5f-4cef-a3f5-4529d6181b77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:38.779147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807a159-ec5f-4cef-a3f5-4529d6181b77 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:38.796470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 335/652] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:38.797265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807a159-ec5f-4cef-a3f5-4529d6181b77 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:38.825992 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:38.866797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fea09-7d13-46ce-aeb2-166ed7c00ced None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:38.913324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe0c0e-8bd1-479a-b6de-a625239fdd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.914600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe0c0e-8bd1-479a-b6de-a625239fdd2c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.914885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe0c0e-8bd1-479a-b6de-a625239fdd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.915160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe0c0e-8bd1-479a-b6de-a625239fdd2c None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.915897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe0c0e-8bd1-479a-b6de-a625239fdd2c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:38.916203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe0c0e-8bd1-479a-b6de-a625239fdd2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:38.917698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe0c0e-8bd1-479a-b6de-a625239fdd2c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:38.924948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807a159-ec5f-4cef-a3f5-4529d6181b77 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:38.948741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 318/653] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:38.953959 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe0c0e-8bd1-479a-b6de-a625239fdd2c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:38.968761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:38.979275 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a77ce-4ee3-42c7-8872-f64ea29293d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:38.980101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a77ce-4ee3-42c7-8872-f64ea29293d6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:38.980565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a77ce-4ee3-42c7-8872-f64ea29293d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:38.980980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a77ce-4ee3-42c7-8872-f64ea29293d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:38.981763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a77ce-4ee3-42c7-8872-f64ea29293d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:38.982244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a77ce-4ee3-42c7-8872-f64ea29293d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:38.983605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a77ce-4ee3-42c7-8872-f64ea29293d6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:38.997872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 336/654] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:39.002946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a77ce-4ee3-42c7-8872-f64ea29293d6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:39.083215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe0c0e-8bd1-479a-b6de-a625239fdd2c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:39.114054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a77ce-4ee3-42c7-8872-f64ea29293d6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:39.122383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:39.135580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff5a36-e0cf-45a6-9ea1-ce1b595d2016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:39.137117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff5a36-e0cf-45a6-9ea1-ce1b595d2016 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:39.137594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff5a36-e0cf-45a6-9ea1-ce1b595d2016 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:39.138341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff5a36-e0cf-45a6-9ea1-ce1b595d2016 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:39.149199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff5a36-e0cf-45a6-9ea1-ce1b595d2016 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:39.150849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff5a36-e0cf-45a6-9ea1-ce1b595d2016 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:39.152760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb005d1f-0763-4769-8eda-ab70b9e68bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:39.153333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb005d1f-0763-4769-8eda-ab70b9e68bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:39.153608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb005d1f-0763-4769-8eda-ab70b9e68bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:39.153875 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb005d1f-0763-4769-8eda-ab70b9e68bc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d6ff5a36-e0cf-45a6-9ea1-ce1b595d2016 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:39.165713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:39.166706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 319/655] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:39.167676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff5a36-e0cf-45a6-9ea1-ce1b595d2016 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:39.228454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:39.248087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb005d1f-0763-4769-8eda-ab70b9e68bc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:39.249607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb005d1f-0763-4769-8eda-ab70b9e68bc5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:39.280410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff5a36-e0cf-45a6-9ea1-ce1b595d2016 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:39.332142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501324a-b2d6-4a83-81d1-8c254a1b5747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:39.332810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501324a-b2d6-4a83-81d1-8c254a1b5747 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:39.333081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501324a-b2d6-4a83-81d1-8c254a1b5747 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:39.333343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501324a-b2d6-4a83-81d1-8c254a1b5747 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:39.342887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501324a-b2d6-4a83-81d1-8c254a1b5747 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:39.344309 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501324a-b2d6-4a83-81d1-8c254a1b5747 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2501324a-b2d6-4a83-81d1-8c254a1b5747 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:39.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:39.357001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:39.357966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:39.357966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:39.357966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:39.357966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:39.357966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:39.357966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:39.357966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:39.357966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:39.357966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 30 19:16:39.357966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:39.361789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 320/656] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:39.362926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501324a-b2d6-4a83-81d1-8c254a1b5747 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:39.379936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 337/657] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:39.381230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb005d1f-0763-4769-8eda-ab70b9e68bc5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:39.430300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:39.490881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501324a-b2d6-4a83-81d1-8c254a1b5747 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:39.505410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:39.518962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb005d1f-0763-4769-8eda-ab70b9e68bc5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:39.535760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7925c-5951-4dd2-bad1-f34fa635f0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:39.536376 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7925c-5951-4dd2-bad1-f34fa635f0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:39.536674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7925c-5951-4dd2-bad1-f34fa635f0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:39.536946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7925c-5951-4dd2-bad1-f34fa635f0b6 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:39.537935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7925c-5951-4dd2-bad1-f34fa635f0b6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:39.538241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7925c-5951-4dd2-bad1-f34fa635f0b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:39.539704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7925c-5951-4dd2-bad1-f34fa635f0b6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:39.554578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5d499-bb2c-4430-9a6d-f8270d86c2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:39.555337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5d499-bb2c-4430-9a6d-f8270d86c2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:39.555705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5d499-bb2c-4430-9a6d-f8270d86c2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:39.556033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5d499-bb2c-4430-9a6d-f8270d86c2a9 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:39.556874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5d499-bb2c-4430-9a6d-f8270d86c2a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:39.557231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5d499-bb2c-4430-9a6d-f8270d86c2a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:39.562744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 321/658] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Sep 30 19:16:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:39.562890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5d499-bb2c-4430-9a6d-f8270d86c2a9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:39.562978 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7925c-5951-4dd2-bad1-f34fa635f0b6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:39.581314 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 338/659] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Sep 30 19:16:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:39.582852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018df79f-7103-4e98-af9f-5e1f03df2aee None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:39.583274 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018df79f-7103-4e98-af9f-5e1f03df2aee None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:39.583700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018df79f-7103-4e98-af9f-5e1f03df2aee None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:39.585058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 339/660] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:39.586749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018df79f-7103-4e98-af9f-5e1f03df2aee None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:39.672044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7925c-5951-4dd2-bad1-f34fa635f0b6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:39.675375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:39.694451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018df79f-7103-4e98-af9f-5e1f03df2aee None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:39.709442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e62171-01a1-41b0-a145-a1bd707368cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:39.710141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e62171-01a1-41b0-a145-a1bd707368cb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:39.710342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e62171-01a1-41b0-a145-a1bd707368cb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:39.710613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e62171-01a1-41b0-a145-a1bd707368cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:39.721633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e62171-01a1-41b0-a145-a1bd707368cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:39.723063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e62171-01a1-41b0-a145-a1bd707368cb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:39.729097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b6d2f-cc35-416d-9860-e1b6617500e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:39.729694 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b6d2f-cc35-416d-9860-e1b6617500e5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:39.729969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b6d2f-cc35-416d-9860-e1b6617500e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:39.730236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b6d2f-cc35-416d-9860-e1b6617500e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e5e62171-01a1-41b0-a145-a1bd707368cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:39.738951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:39.739903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:39.740868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:39.740868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:39.740868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:39.740868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 322/661] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:39.741163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e62171-01a1-41b0-a145-a1bd707368cb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:39.751857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:39.807778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b6d2f-cc35-416d-9860-e1b6617500e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:39.809248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b6d2f-cc35-416d-9860-e1b6617500e5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:39.846409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e62171-01a1-41b0-a145-a1bd707368cb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:39.881219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f0b51-5614-4547-8dc9-1dca14f38c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:39.881918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f0b51-5614-4547-8dc9-1dca14f38c25 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:39.882207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f0b51-5614-4547-8dc9-1dca14f38c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:39.882474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f0b51-5614-4547-8dc9-1dca14f38c25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:39.904902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 340/662] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:39.906275 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f9bb5-e89c-4de7-9801-55c724e8167e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:39.906620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f9bb5-e89c-4de7-9801-55c724e8167e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:39.906954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f9bb5-e89c-4de7-9801-55c724e8167e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:39.913370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:39.926128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-426f9bb5-e89c-4de7-9801-55c724e8167e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:39.927302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-426f9bb5-e89c-4de7-9801-55c724e8167e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:39.959046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f0b51-5614-4547-8dc9-1dca14f38c25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:39.961503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f0b51-5614-4547-8dc9-1dca14f38c25 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:39.975751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:40.009443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 341/663] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:40.009443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f9bb5-e89c-4de7-9801-55c724e8167e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.082261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 323/664] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:40.083270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f0b51-5614-4547-8dc9-1dca14f38c25 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.135719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f9bb5-e89c-4de7-9801-55c724e8167e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:40.140230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:40.188420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6fe0a-352f-4b1e-831e-851ecf6c90d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:40.188991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f0b51-5614-4547-8dc9-1dca14f38c25 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:40.189123 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6fe0a-352f-4b1e-831e-851ecf6c90d4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:40.189291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6fe0a-352f-4b1e-831e-851ecf6c90d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:40.189562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6fe0a-352f-4b1e-831e-851ecf6c90d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:40.190201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6fe0a-352f-4b1e-831e-851ecf6c90d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:40.190500 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6fe0a-352f-4b1e-831e-851ecf6c90d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:40.196527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6fe0a-352f-4b1e-831e-851ecf6c90d4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:40.211906 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 342/665] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:40.212739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6fe0a-352f-4b1e-831e-851ecf6c90d4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.230397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b94bd-52e0-46ea-b4fe-27f4aef4c6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:40.231139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b94bd-52e0-46ea-b4fe-27f4aef4c6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:40.231477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b94bd-52e0-46ea-b4fe-27f4aef4c6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:40.231830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b94bd-52e0-46ea-b4fe-27f4aef4c6e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:40.242753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b94bd-52e0-46ea-b4fe-27f4aef4c6e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:40.244356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b94bd-52e0-46ea-b4fe-27f4aef4c6e4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:40.255344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c14b94bd-52e0-46ea-b4fe-27f4aef4c6e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:40.260445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:40.261616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:40.262807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:40.262807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:40.262807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:40.263441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 324/666] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:40.264403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b94bd-52e0-46ea-b4fe-27f4aef4c6e4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.328282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6fe0a-352f-4b1e-831e-851ecf6c90d4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:40.365646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83e09a-26fa-4fd2-836f-678277d828da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:40.366533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83e09a-26fa-4fd2-836f-678277d828da None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:40.366819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83e09a-26fa-4fd2-836f-678277d828da None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:40.367113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83e09a-26fa-4fd2-836f-678277d828da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:40.374991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b94bd-52e0-46ea-b4fe-27f4aef4c6e4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:40.378518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83e09a-26fa-4fd2-836f-678277d828da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:40.379944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83e09a-26fa-4fd2-836f-678277d828da None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:40.389331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-4e83e09a-26fa-4fd2-836f-678277d828da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:40.394905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:40.395865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:40.396842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:40.396842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:40.396842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:40.396842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 343/667] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:40.400925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83e09a-26fa-4fd2-836f-678277d828da None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.419666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d903f0-b045-474a-867e-25e4a5ed9a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:40.420292 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d903f0-b045-474a-867e-25e4a5ed9a7c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:40.420634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d903f0-b045-474a-867e-25e4a5ed9a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:40.420888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d903f0-b045-474a-867e-25e4a5ed9a7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:40.427496 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d903f0-b045-474a-867e-25e4a5ed9a7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:40.428966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d903f0-b045-474a-867e-25e4a5ed9a7c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-69d903f0-b045-474a-867e-25e4a5ed9a7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:40.440663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:40.441578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:40.442504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:40.442504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:40.442504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:40.442504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:40.442504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:40.442504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:40.442504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:40.442504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:40.444688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 325/668] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:40.445741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d903f0-b045-474a-867e-25e4a5ed9a7c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.502342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:40.515451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83e09a-26fa-4fd2-836f-678277d828da None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:40.550097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3f9a5-39c9-40ac-bfd8-18ab64def087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:40.550703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3f9a5-39c9-40ac-bfd8-18ab64def087 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:40.550980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3f9a5-39c9-40ac-bfd8-18ab64def087 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:40.551250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3f9a5-39c9-40ac-bfd8-18ab64def087 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:40.551524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d903f0-b045-474a-867e-25e4a5ed9a7c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:40.551893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3f9a5-39c9-40ac-bfd8-18ab64def087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:40.552191 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3f9a5-39c9-40ac-bfd8-18ab64def087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:40.553649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3f9a5-39c9-40ac-bfd8-18ab64def087 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:40.566590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 344/669] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:40.567243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3f9a5-39c9-40ac-bfd8-18ab64def087 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.592986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d8063-99a6-4bdc-8764-9092cd937172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:40.593814 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d8063-99a6-4bdc-8764-9092cd937172 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:40.594258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d8063-99a6-4bdc-8764-9092cd937172 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:40.594702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d8063-99a6-4bdc-8764-9092cd937172 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:40.611045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d8063-99a6-4bdc-8764-9092cd937172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7e8d8063-99a6-4bdc-8764-9092cd937172 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:40.619861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:40.620828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 326/670] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:40.625174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d8063-99a6-4bdc-8764-9092cd937172 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.656736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:40.695317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3f9a5-39c9-40ac-bfd8-18ab64def087 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:40.735795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca63a9-a9e8-4db5-8e40-5a7a7a028b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:40.735989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d8063-99a6-4bdc-8764-9092cd937172 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:40.736860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca63a9-a9e8-4db5-8e40-5a7a7a028b17 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:40.737350 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca63a9-a9e8-4db5-8e40-5a7a7a028b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:40.737724 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca63a9-a9e8-4db5-8e40-5a7a7a028b17 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:40.738972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca63a9-a9e8-4db5-8e40-5a7a7a028b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:40.738972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca63a9-a9e8-4db5-8e40-5a7a7a028b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:40.740104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca63a9-a9e8-4db5-8e40-5a7a7a028b17 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:40.756338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 345/671] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:40.756338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca63a9-a9e8-4db5-8e40-5a7a7a028b17 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.766477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:40.778247 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dd8d6-9fe5-4dc1-89f2-a99982117a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:40.778855 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dd8d6-9fe5-4dc1-89f2-a99982117a88 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:40.779128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dd8d6-9fe5-4dc1-89f2-a99982117a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:40.779396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dd8d6-9fe5-4dc1-89f2-a99982117a88 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:40.780084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dd8d6-9fe5-4dc1-89f2-a99982117a88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:40.780398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dd8d6-9fe5-4dc1-89f2-a99982117a88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:40.781837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dd8d6-9fe5-4dc1-89f2-a99982117a88 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:40.794733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 327/672] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:40.795448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dd8d6-9fe5-4dc1-89f2-a99982117a88 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.865800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca63a9-a9e8-4db5-8e40-5a7a7a028b17 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:40.890964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:40.902040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca9b4d-993c-44dd-98af-fdd6cba28cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:40.902822 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca9b4d-993c-44dd-98af-fdd6cba28cbe None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:40.903098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca9b4d-993c-44dd-98af-fdd6cba28cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:40.903365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca9b4d-993c-44dd-98af-fdd6cba28cbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:40.904166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dd8d6-9fe5-4dc1-89f2-a99982117a88 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:40.910184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca9b4d-993c-44dd-98af-fdd6cba28cbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:40.911624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca9b4d-993c-44dd-98af-fdd6cba28cbe None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-41ca9b4d-993c-44dd-98af-fdd6cba28cbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:40.924860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:40.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:40.927127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:40.927127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:40.927127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:40.927127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:40.927127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:40.927127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:40.927127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:40.927127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:40.927127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 346/673] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:40.927127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca9b4d-993c-44dd-98af-fdd6cba28cbe None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.941735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506b9b5-ba92-4b02-806d-98d69c6cf6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:40.942329 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506b9b5-ba92-4b02-806d-98d69c6cf6ed None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:40.942600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506b9b5-ba92-4b02-806d-98d69c6cf6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:40.942865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506b9b5-ba92-4b02-806d-98d69c6cf6ed None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:40.943521 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506b9b5-ba92-4b02-806d-98d69c6cf6ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:40.943818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506b9b5-ba92-4b02-806d-98d69c6cf6ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:40.946046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506b9b5-ba92-4b02-806d-98d69c6cf6ed None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:40.953988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 328/674] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:40.954749 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506b9b5-ba92-4b02-806d-98d69c6cf6ed None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:40.992285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:41.025758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca9b4d-993c-44dd-98af-fdd6cba28cbe None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:41.053136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506b9b5-ba92-4b02-806d-98d69c6cf6ed None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:41.057901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d7036-6c0e-4d4f-81cc-cb26a9813579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:41.058522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7036-6c0e-4d4f-81cc-cb26a9813579 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.058796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7036-6c0e-4d4f-81cc-cb26a9813579 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.059063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d7036-6c0e-4d4f-81cc-cb26a9813579 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:41.065549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d7036-6c0e-4d4f-81cc-cb26a9813579 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:41.066984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d7036-6c0e-4d4f-81cc-cb26a9813579 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-222d7036-6c0e-4d4f-81cc-cb26a9813579 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:41.078391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:41.079430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:41.080353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:41.080353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:41.080353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:41.080353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:41.080353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:41.080353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:41.080353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:41.080353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:41.080353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 347/675] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:41.081299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d5a5f-0fa3-4e70-87b2-293defc4266e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.081299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d5a5f-0fa3-4e70-87b2-293defc4266e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.081593 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d5a5f-0fa3-4e70-87b2-293defc4266e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:41.082789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 348/676] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:41.083476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d5a5f-0fa3-4e70-87b2-293defc4266e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:41.093822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6748a-c37d-4a74-84ea-a6f2847b3fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:41.094431 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6748a-c37d-4a74-84ea-a6f2847b3fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.094710 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6748a-c37d-4a74-84ea-a6f2847b3fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.094981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6748a-c37d-4a74-84ea-a6f2847b3fa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:41.095706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6748a-c37d-4a74-84ea-a6f2847b3fa7 None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=65d3020e1f6c414caabbaa185fb7ed48) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:41.096010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6748a-c37d-4a74-84ea-a6f2847b3fa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:41.097464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6748a-c37d-4a74-84ea-a6f2847b3fa7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:41.103567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 329/677] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:40 2022] GET /identity/v3/role_assignments?user.id=65d3020e1f6c414caabbaa185fb7ed48&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:41.104327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6748a-c37d-4a74-84ea-a6f2847b3fa7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:41.138473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:41.200449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d5a5f-0fa3-4e70-87b2-293defc4266e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:41.228883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:41.234485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6748a-c37d-4a74-84ea-a6f2847b3fa7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:41.238156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01735c37-a388-4a8c-bce5-a1b75aca240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:41.238880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01735c37-a388-4a8c-bce5-a1b75aca240b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.239251 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01735c37-a388-4a8c-bce5-a1b75aca240b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.239614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01735c37-a388-4a8c-bce5-a1b75aca240b None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:41.240433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01735c37-a388-4a8c-bce5-a1b75aca240b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:41.240852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01735c37-a388-4a8c-bce5-a1b75aca240b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:41.242631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01735c37-a388-4a8c-bce5-a1b75aca240b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:41.262306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 349/678] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:41.262306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01735c37-a388-4a8c-bce5-a1b75aca240b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:41.272338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1fd8e-a497-4d90-9610-92cd05a7e1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:41.272993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1fd8e-a497-4d90-9610-92cd05a7e1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.273274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1fd8e-a497-4d90-9610-92cd05a7e1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.274125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1fd8e-a497-4d90-9610-92cd05a7e1a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:41.287773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1fd8e-a497-4d90-9610-92cd05a7e1a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:41.289258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1fd8e-a497-4d90-9610-92cd05a7e1a9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2be1fd8e-a497-4d90-9610-92cd05a7e1a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:41.304514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:41.306201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:41.307171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:41.307171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:41.307171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:41.307171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 330/679] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:41.308126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeeb59f-4122-4add-b5bd-10b93286f221 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.308426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeeb59f-4122-4add-b5bd-10b93286f221 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.308699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeeb59f-4122-4add-b5bd-10b93286f221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:41.328032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2aeeb59f-4122-4add-b5bd-10b93286f221 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:41.329269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2aeeb59f-4122-4add-b5bd-10b93286f221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:41.382172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01735c37-a388-4a8c-bce5-a1b75aca240b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:41.393784 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:41.404267 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 331/680] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:41.405133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeeb59f-4122-4add-b5bd-10b93286f221 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:41.424949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382df10-05fe-4b80-a61d-beec42eb8932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:41.425971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382df10-05fe-4b80-a61d-beec42eb8932 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.426518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382df10-05fe-4b80-a61d-beec42eb8932 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.427030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382df10-05fe-4b80-a61d-beec42eb8932 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:41.428063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382df10-05fe-4b80-a61d-beec42eb8932 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:41.428586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382df10-05fe-4b80-a61d-beec42eb8932 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:41.430854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382df10-05fe-4b80-a61d-beec42eb8932 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:41.448774 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 350/681] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:41.449902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382df10-05fe-4b80-a61d-beec42eb8932 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:41.502867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:41.530096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeeb59f-4122-4add-b5bd-10b93286f221 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:41.565507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382df10-05fe-4b80-a61d-beec42eb8932 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:41.568073 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e6b6f-48be-4d65-87b5-a9d7564ba3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:41.568978 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6b6f-48be-4d65-87b5-a9d7564ba3ec None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.569507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6b6f-48be-4d65-87b5-a9d7564ba3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.570038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6b6f-48be-4d65-87b5-a9d7564ba3ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:41.571520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e6b6f-48be-4d65-87b5-a9d7564ba3ec None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=18ea49d1f6c040f18511535b7e51a53b) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:41.572045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e6b6f-48be-4d65-87b5-a9d7564ba3ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:41.573905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e6b6f-48be-4d65-87b5-a9d7564ba3ec None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:41.581987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 332/682] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:41 2022] GET /identity/v3/role_assignments?user.id=18ea49d1f6c040f18511535b7e51a53b&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:41.583750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e6b6f-48be-4d65-87b5-a9d7564ba3ec None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:41.615709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb672e-2e9e-4650-b877-3ab792eabf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:41.616338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb672e-2e9e-4650-b877-3ab792eabf2e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.616639 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb672e-2e9e-4650-b877-3ab792eabf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.616911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb672e-2e9e-4650-b877-3ab792eabf2e None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:41.617693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb672e-2e9e-4650-b877-3ab792eabf2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:41.617991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb672e-2e9e-4650-b877-3ab792eabf2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:41.619530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb672e-2e9e-4650-b877-3ab792eabf2e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:41.634715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 351/683] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:41.641557 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb672e-2e9e-4650-b877-3ab792eabf2e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:41.722720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:41.771580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e6b6f-48be-4d65-87b5-a9d7564ba3ec None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:41.785065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb672e-2e9e-4650-b877-3ab792eabf2e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:41.796902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:41.823915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80127ac5-bca0-4549-94f5-9ccb67a2e782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:41.824467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80127ac5-bca0-4549-94f5-9ccb67a2e782 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.824745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80127ac5-bca0-4549-94f5-9ccb67a2e782 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.825021 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80127ac5-bca0-4549-94f5-9ccb67a2e782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:41.826606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80127ac5-bca0-4549-94f5-9ccb67a2e782 None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=58a9b371434243daa4176215daefb79c) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:41.826606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80127ac5-bca0-4549-94f5-9ccb67a2e782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:41.827997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80127ac5-bca0-4549-94f5-9ccb67a2e782 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:41.833440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 333/684] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:41 2022] GET /identity/v3/role_assignments?user.id=58a9b371434243daa4176215daefb79c&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:41.834230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80127ac5-bca0-4549-94f5-9ccb67a2e782 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:41.836414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45424082-8c5a-4dd3-8eee-bba45ec8817b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:41.837386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45424082-8c5a-4dd3-8eee-bba45ec8817b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.837895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45424082-8c5a-4dd3-8eee-bba45ec8817b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.838348 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45424082-8c5a-4dd3-8eee-bba45ec8817b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:41.847110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45424082-8c5a-4dd3-8eee-bba45ec8817b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:41.849083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45424082-8c5a-4dd3-8eee-bba45ec8817b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-45424082-8c5a-4dd3-8eee-bba45ec8817b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:41.863629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:41.864721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:41.865738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:41.865738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:41.865738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:41.865738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:41.865738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:41.865738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:41.865738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:41.865738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:41.865738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:41.865738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:41.868906 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 352/685] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:41.874786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45424082-8c5a-4dd3-8eee-bba45ec8817b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:41.947174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80127ac5-bca0-4549-94f5-9ccb67a2e782 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:41.980777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:41.986269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773b35a-846a-407b-a5c2-0bbc107c312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:41.986957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773b35a-846a-407b-a5c2-0bbc107c312f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:41.987238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773b35a-846a-407b-a5c2-0bbc107c312f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:41.987507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773b35a-846a-407b-a5c2-0bbc107c312f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:42.000118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773b35a-846a-407b-a5c2-0bbc107c312f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6773b35a-846a-407b-a5c2-0bbc107c312f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:42.005219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:42.006234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:42.007141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 334/686] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:42.012839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773b35a-846a-407b-a5c2-0bbc107c312f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:42.107739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45424082-8c5a-4dd3-8eee-bba45ec8817b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:42.126394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773b35a-846a-407b-a5c2-0bbc107c312f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:42.150445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae73473-641e-4614-b0d0-7ea9a0cdad3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:42.151408 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae73473-641e-4614-b0d0-7ea9a0cdad3e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:42.151911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae73473-641e-4614-b0d0-7ea9a0cdad3e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:42.152422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae73473-641e-4614-b0d0-7ea9a0cdad3e None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:42.153350 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae73473-641e-4614-b0d0-7ea9a0cdad3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:42.153878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae73473-641e-4614-b0d0-7ea9a0cdad3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:42.155497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae73473-641e-4614-b0d0-7ea9a0cdad3e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:42.164568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f59818-e617-4c53-9040-5727db73f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:42.165286 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f59818-e617-4c53-9040-5727db73f677 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:42.165578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f59818-e617-4c53-9040-5727db73f677 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:42.165832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f59818-e617-4c53-9040-5727db73f677 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:42.166496 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f59818-e617-4c53-9040-5727db73f677 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:42.166792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f59818-e617-4c53-9040-5727db73f677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:42.169259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f59818-e617-4c53-9040-5727db73f677 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:42.179037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 335/687] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:42.180404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 353/688] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 30 19:16:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:42.185431 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f59818-e617-4c53-9040-5727db73f677 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:42.186528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae73473-641e-4614-b0d0-7ea9a0cdad3e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:42.221684 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:42.244986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:42.294619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f59818-e617-4c53-9040-5727db73f677 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:42.316154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae73473-641e-4614-b0d0-7ea9a0cdad3e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:42.332347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca9d57-1691-453e-8212-8cf5a3c3656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:42.333085 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca9d57-1691-453e-8212-8cf5a3c3656d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:42.333425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca9d57-1691-453e-8212-8cf5a3c3656d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:42.333756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca9d57-1691-453e-8212-8cf5a3c3656d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:42.348783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca9d57-1691-453e-8212-8cf5a3c3656d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:42.350468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca9d57-1691-453e-8212-8cf5a3c3656d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:42.358011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa016b44-5aa2-48f9-bbe5-bd57cd641be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:42.358571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa016b44-5aa2-48f9-bbe5-bd57cd641be6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:42.358851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa016b44-5aa2-48f9-bbe5-bd57cd641be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:42.359122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa016b44-5aa2-48f9-bbe5-bd57cd641be6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b3ca9d57-1691-453e-8212-8cf5a3c3656d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:42.369224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:42.370227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:42.371144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:42.371144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:42.371144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:42.371144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 336/689] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:42.377616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca9d57-1691-453e-8212-8cf5a3c3656d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:42.458477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa016b44-5aa2-48f9-bbe5-bd57cd641be6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:42.459924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa016b44-5aa2-48f9-bbe5-bd57cd641be6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:42.502658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:42.507557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca9d57-1691-453e-8212-8cf5a3c3656d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:42.526767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:42.565669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f2030-0b0a-4b6e-aa2f-54ccf0d82bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:42.566668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f2030-0b0a-4b6e-aa2f-54ccf0d82bca None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:42.567194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f2030-0b0a-4b6e-aa2f-54ccf0d82bca None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:42.567656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f2030-0b0a-4b6e-aa2f-54ccf0d82bca None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:42.568726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f2030-0b0a-4b6e-aa2f-54ccf0d82bca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:42.569247 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f2030-0b0a-4b6e-aa2f-54ccf0d82bca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:42.571121 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f2030-0b0a-4b6e-aa2f-54ccf0d82bca None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:42.585277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 337/690] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:42.591370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f2030-0b0a-4b6e-aa2f-54ccf0d82bca None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:42.672141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 354/691] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:42.680727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa016b44-5aa2-48f9-bbe5-bd57cd641be6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:42.775077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f2030-0b0a-4b6e-aa2f-54ccf0d82bca None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:42.823234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c43e48-ebc7-4e4b-b343-81cd4c783f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:42.824210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c43e48-ebc7-4e4b-b343-81cd4c783f2e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:42.824829 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c43e48-ebc7-4e4b-b343-81cd4c783f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:42.825348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c43e48-ebc7-4e4b-b343-81cd4c783f2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:42.834129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c43e48-ebc7-4e4b-b343-81cd4c783f2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:42.836360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c43e48-ebc7-4e4b-b343-81cd4c783f2e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-89c43e48-ebc7-4e4b-b343-81cd4c783f2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:42.848985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:42.850030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:42.850979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:42.850979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:42.850979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:42.850979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:42.850979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:42.850979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:42.850979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:42.850979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:42.850979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:42.850979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:42.854006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 338/692] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 30 19:16:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:42.860006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c43e48-ebc7-4e4b-b343-81cd4c783f2e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:42.874600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa016b44-5aa2-48f9-bbe5-bd57cd641be6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:42.882263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:42.936741 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:42.947750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8cb9d-98b3-4634-a749-423b0aed019f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:42.949415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8cb9d-98b3-4634-a749-423b0aed019f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:42.951018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8cb9d-98b3-4634-a749-423b0aed019f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:42.952637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8cb9d-98b3-4634-a749-423b0aed019f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:42.972069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8cb9d-98b3-4634-a749-423b0aed019f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:42.978654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8cb9d-98b3-4634-a749-423b0aed019f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-09c8cb9d-98b3-4634-a749-423b0aed019f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:42.998619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:42.999651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:43.000619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:43.000619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:43.000619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:43.002996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 355/693] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:43.009183 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8cb9d-98b3-4634-a749-423b0aed019f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:43.043004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c43e48-ebc7-4e4b-b343-81cd4c783f2e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:43.094442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979a824-6c50-481d-9c6d-83e955f97120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:43.095613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979a824-6c50-481d-9c6d-83e955f97120 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:43.096100 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979a824-6c50-481d-9c6d-83e955f97120 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:43.096609 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979a824-6c50-481d-9c6d-83e955f97120 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:43.097943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979a824-6c50-481d-9c6d-83e955f97120 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:43.098461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979a824-6c50-481d-9c6d-83e955f97120 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:43.100546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979a824-6c50-481d-9c6d-83e955f97120 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:43.121940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 339/694] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 30 19:16:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:43.127619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979a824-6c50-481d-9c6d-83e955f97120 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:43.203794 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8cb9d-98b3-4634-a749-423b0aed019f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:43.242010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2a99b-79a1-4f5c-b70e-4960106db02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:43.243952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2a99b-79a1-4f5c-b70e-4960106db02d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:43.244601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2a99b-79a1-4f5c-b70e-4960106db02d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:43.245189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2a99b-79a1-4f5c-b70e-4960106db02d None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:43.246036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2a99b-79a1-4f5c-b70e-4960106db02d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:43.246540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2a99b-79a1-4f5c-b70e-4960106db02d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:43.248157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2a99b-79a1-4f5c-b70e-4960106db02d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:43.262190 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 356/695] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:43.273460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2a99b-79a1-4f5c-b70e-4960106db02d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:43.275995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:43.316850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979a824-6c50-481d-9c6d-83e955f97120 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:43.361415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:43.427906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ecf512-0f02-4171-834d-1154d71866f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:43.428911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecf512-0f02-4171-834d-1154d71866f4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:43.429390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecf512-0f02-4171-834d-1154d71866f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:43.430423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecf512-0f02-4171-834d-1154d71866f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:43.542346 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2a99b-79a1-4f5c-b70e-4960106db02d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:43.559832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ecf512-0f02-4171-834d-1154d71866f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:43.561764 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ecf512-0f02-4171-834d-1154d71866f4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:43.608402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d842d2-615c-4660-9ed9-9648c951e3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:43.608402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d842d2-615c-4660-9ed9-9648c951e3da None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:43.608402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d842d2-615c-4660-9ed9-9648c951e3da None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:43.608402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d842d2-615c-4660-9ed9-9648c951e3da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:43.619538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d842d2-615c-4660-9ed9-9648c951e3da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:43.621196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d842d2-615c-4660-9ed9-9648c951e3da None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-10d842d2-615c-4660-9ed9-9648c951e3da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:43.632792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:43.633873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:43.634821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:43.634821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:43.634821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:43.634821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:43.634821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:43.634821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:43.634821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:43.634821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:43.639796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 357/696] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:43.641496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08035d98-0c68-4fd6-8dbd-bf216870d741 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:43.641608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08035d98-0c68-4fd6-8dbd-bf216870d741 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:43.641884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08035d98-0c68-4fd6-8dbd-bf216870d741 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:43.643830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 358/697] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:43.644819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08035d98-0c68-4fd6-8dbd-bf216870d741 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:43.653802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:43.690348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 340/698] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:43.691743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66ef8f-dca6-40a9-af01-7666fcbdb56d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:43.692135 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66ef8f-dca6-40a9-af01-7666fcbdb56d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:43.692541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66ef8f-dca6-40a9-af01-7666fcbdb56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:43.710072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8a66ef8f-dca6-40a9-af01-7666fcbdb56d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:43.711889 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8a66ef8f-dca6-40a9-af01-7666fcbdb56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:43.759159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:43.760595 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08035d98-0c68-4fd6-8dbd-bf216870d741 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:43.787682 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 341/699] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:43.788814 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66ef8f-dca6-40a9-af01-7666fcbdb56d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:43.817157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e0e54-d022-4186-8f24-2f4911d46ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:43.817691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e0e54-d022-4186-8f24-2f4911d46ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:43.817965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e0e54-d022-4186-8f24-2f4911d46ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:43.818230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e0e54-d022-4186-8f24-2f4911d46ab0 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:43.819039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e0e54-d022-4186-8f24-2f4911d46ab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:43.819321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e0e54-d022-4186-8f24-2f4911d46ab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:43.820920 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e0e54-d022-4186-8f24-2f4911d46ab0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:43.852487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 359/700] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:43.860396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e0e54-d022-4186-8f24-2f4911d46ab0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:43.931899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66ef8f-dca6-40a9-af01-7666fcbdb56d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:43.945819 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:44.003591 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c4761-2b6c-4846-a8ee-a6b02597a6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:44.004310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c4761-2b6c-4846-a8ee-a6b02597a6ac None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.004608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c4761-2b6c-4846-a8ee-a6b02597a6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.004873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c4761-2b6c-4846-a8ee-a6b02597a6ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.026497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c4761-2b6c-4846-a8ee-a6b02597a6ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:44.028009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c4761-2b6c-4846-a8ee-a6b02597a6ac None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:44.045669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e0e54-d022-4186-8f24-2f4911d46ab0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e36c4761-2b6c-4846-a8ee-a6b02597a6ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:44.060526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:44.061606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:44.063011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:44.063011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:44.063011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:44.064164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 342/701] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:44.067516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c4761-2b6c-4846-a8ee-a6b02597a6ac None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:44.105284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06b87e-afdb-491b-95f9-050c353386f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:44.105885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06b87e-afdb-491b-95f9-050c353386f9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.106156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06b87e-afdb-491b-95f9-050c353386f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.106419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06b87e-afdb-491b-95f9-050c353386f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.107128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06b87e-afdb-491b-95f9-050c353386f9 None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=b8785c8c5ac84393afcaf88e891f3b7a) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:44.107433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06b87e-afdb-491b-95f9-050c353386f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:44.108892 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06b87e-afdb-491b-95f9-050c353386f9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:44.115119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 360/702] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:43 2022] GET /identity/v3/role_assignments?user.id=b8785c8c5ac84393afcaf88e891f3b7a&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:44.118561 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06b87e-afdb-491b-95f9-050c353386f9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:44.163751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:44.239464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06b87e-afdb-491b-95f9-050c353386f9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:44.253216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c4761-2b6c-4846-a8ee-a6b02597a6ac None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:44.278195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05881512-4644-4b02-9bcd-8a39c760ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:44.279016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05881512-4644-4b02-9bcd-8a39c760ad22 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.279386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05881512-4644-4b02-9bcd-8a39c760ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.279752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05881512-4644-4b02-9bcd-8a39c760ad22 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.280478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05881512-4644-4b02-9bcd-8a39c760ad22 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:44.281980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05881512-4644-4b02-9bcd-8a39c760ad22 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-05881512-4644-4b02-9bcd-8a39c760ad22 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:44.295723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:44.296940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:44.298074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 361/703] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Sep 30 19:16:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:44.303854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05881512-4644-4b02-9bcd-8a39c760ad22 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:44.345295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc52e2-caed-408f-b054-eb9b982f7a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:44.346208 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc52e2-caed-408f-b054-eb9b982f7a46 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.346664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc52e2-caed-408f-b054-eb9b982f7a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.347118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc52e2-caed-408f-b054-eb9b982f7a46 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.347965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc52e2-caed-408f-b054-eb9b982f7a46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:44.348484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc52e2-caed-408f-b054-eb9b982f7a46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:44.350610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc52e2-caed-408f-b054-eb9b982f7a46 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:44.368935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 343/704] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:44.374393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc52e2-caed-408f-b054-eb9b982f7a46 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:44.402821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:44.428836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:44.449970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05881512-4644-4b02-9bcd-8a39c760ad22 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:44.497029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611afead-3144-4928-bd9d-2931b2b9fe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:44.497999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611afead-3144-4928-bd9d-2931b2b9fe12 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.498483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611afead-3144-4928-bd9d-2931b2b9fe12 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.498988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611afead-3144-4928-bd9d-2931b2b9fe12 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.499835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611afead-3144-4928-bd9d-2931b2b9fe12 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:44.500384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611afead-3144-4928-bd9d-2931b2b9fe12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:44.501958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611afead-3144-4928-bd9d-2931b2b9fe12 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:44.506972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 362/705] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 30 19:16:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:44.508465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add97c03-b481-4140-b10a-c84d5ba58639 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.509044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add97c03-b481-4140-b10a-c84d5ba58639 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.509536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add97c03-b481-4140-b10a-c84d5ba58639 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.511159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 363/706] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:44.512586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add97c03-b481-4140-b10a-c84d5ba58639 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:44.520459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc52e2-caed-408f-b054-eb9b982f7a46 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:44.560228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05210d4-d8ff-4564-9064-b90acb7bab2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:44.560888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05210d4-d8ff-4564-9064-b90acb7bab2d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.561160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05210d4-d8ff-4564-9064-b90acb7bab2d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.561423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05210d4-d8ff-4564-9064-b90acb7bab2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.569397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05210d4-d8ff-4564-9064-b90acb7bab2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:44.570887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05210d4-d8ff-4564-9064-b90acb7bab2d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e05210d4-d8ff-4564-9064-b90acb7bab2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:44.582413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:44.583547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:44.584683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:44.584683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:44.584683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:44.584683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:44.584683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:44.584683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:44.584683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:44.584683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:44.584683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 344/707] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:44.585221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacdcb0-a141-45ea-84f1-d9f24271ce1a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.585221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacdcb0-a141-45ea-84f1-d9f24271ce1a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.585615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacdcb0-a141-45ea-84f1-d9f24271ce1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.610405 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dfacdcb0-a141-45ea-84f1-d9f24271ce1a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:44.612149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dfacdcb0-a141-45ea-84f1-d9f24271ce1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:44.627258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add97c03-b481-4140-b10a-c84d5ba58639 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:44.663235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d59b1-0b15-4996-a6f2-bf69f5c7bf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:44.663841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d59b1-0b15-4996-a6f2-bf69f5c7bf7e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.664152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d59b1-0b15-4996-a6f2-bf69f5c7bf7e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.664444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d59b1-0b15-4996-a6f2-bf69f5c7bf7e None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.665074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d59b1-0b15-4996-a6f2-bf69f5c7bf7e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:44.665374 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d59b1-0b15-4996-a6f2-bf69f5c7bf7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:44.666728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d59b1-0b15-4996-a6f2-bf69f5c7bf7e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:44.672121 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 364/708] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 30 19:16:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:44.673293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480921c0-1973-4150-b7e6-f3a133af8f97 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.673570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480921c0-1973-4150-b7e6-f3a133af8f97 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.673864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480921c0-1973-4150-b7e6-f3a133af8f97 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.676493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 365/709] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:44.676493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480921c0-1973-4150-b7e6-f3a133af8f97 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:44.695681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:44.706315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 345/710] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:44.708010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f602-ab91-4e2e-b32a-8d44bfdaf00d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.708471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f602-ab91-4e2e-b32a-8d44bfdaf00d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.708914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f602-ab91-4e2e-b32a-8d44bfdaf00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.750082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6fe5f602-ab91-4e2e-b32a-8d44bfdaf00d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:44.751550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6fe5f602-ab91-4e2e-b32a-8d44bfdaf00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:44.754007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:44.804124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480921c0-1973-4150-b7e6-f3a133af8f97 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:44.861579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d435c-9c31-4fad-8e9d-9277d8d5e555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:44.862538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d435c-9c31-4fad-8e9d-9277d8d5e555 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:44.863092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d435c-9c31-4fad-8e9d-9277d8d5e555 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:44.863564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d435c-9c31-4fad-8e9d-9277d8d5e555 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:44.864526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d435c-9c31-4fad-8e9d-9277d8d5e555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:44.865039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d435c-9c31-4fad-8e9d-9277d8d5e555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:44.867387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d435c-9c31-4fad-8e9d-9277d8d5e555 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:44.895999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 366/711] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:44.903684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 346/712] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:44.908459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d435c-9c31-4fad-8e9d-9277d8d5e555 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:44.985761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:45.019343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d435c-9c31-4fad-8e9d-9277d8d5e555 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:45.037952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5f602-ab91-4e2e-b32a-8d44bfdaf00d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:45.065924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9659132-1055-476c-a180-05d464ab2fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:45.066533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9659132-1055-476c-a180-05d464ab2fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:45.066805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9659132-1055-476c-a180-05d464ab2fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:45.067073 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9659132-1055-476c-a180-05d464ab2fc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:45.067792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9659132-1055-476c-a180-05d464ab2fc3 None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=5952db29e6fc4ddbb9aeb0522497312c) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:45.068093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9659132-1055-476c-a180-05d464ab2fc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:45.069558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9659132-1055-476c-a180-05d464ab2fc3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:45.080406 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 367/713] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:44 2022] GET /identity/v3/role_assignments?user.id=5952db29e6fc4ddbb9aeb0522497312c&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:45.156551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5f602-ab91-4e2e-b32a-8d44bfdaf00d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:45.194060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b14cc3-1974-44df-b317-47450c21d6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:45.195135 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b14cc3-1974-44df-b317-47450c21d6db None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:45.195620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b14cc3-1974-44df-b317-47450c21d6db None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:45.196118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b14cc3-1974-44df-b317-47450c21d6db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:45.203682 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:45.208163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b14cc3-1974-44df-b317-47450c21d6db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:45.209695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b14cc3-1974-44df-b317-47450c21d6db None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:45.223228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9659132-1055-476c-a180-05d464ab2fc3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-51b14cc3-1974-44df-b317-47450c21d6db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:45.236203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:45.237377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:45.238291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:45.238291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:45.238291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:45.238291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 347/714] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:45.243828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b14cc3-1974-44df-b317-47450c21d6db None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:45.421820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9659132-1055-476c-a180-05d464ab2fc3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:45.427558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b14cc3-1974-44df-b317-47450c21d6db None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:45.466265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e0f40-841a-49a6-bd77-61d33b730db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:45.466875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e0f40-841a-49a6-bd77-61d33b730db6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:45.467154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e0f40-841a-49a6-bd77-61d33b730db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:45.467424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e0f40-841a-49a6-bd77-61d33b730db6 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:45.468494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e0f40-841a-49a6-bd77-61d33b730db6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:45.468809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e0f40-841a-49a6-bd77-61d33b730db6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:45.470175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e0f40-841a-49a6-bd77-61d33b730db6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:45.484299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 348/715] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:45.489478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e0f40-841a-49a6-bd77-61d33b730db6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:45.511467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d40b1-f736-4154-9610-a2bd9b786fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:45.512138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d40b1-f736-4154-9610-a2bd9b786fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:45.512392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d40b1-f736-4154-9610-a2bd9b786fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:45.512665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d40b1-f736-4154-9610-a2bd9b786fa7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:45.513244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d40b1-f736-4154-9610-a2bd9b786fa7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:45.516226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d40b1-f736-4154-9610-a2bd9b786fa7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-0a7d40b1-f736-4154-9610-a2bd9b786fa7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:45.535244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:45.536281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 19:16:45.537162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:45.538164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 368/716] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:45 2022] GET /identity/v3/services/network => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:45.543097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d40b1-f736-4154-9610-a2bd9b786fa7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:45.546874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:45.586004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:45.694062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d40b1-f736-4154-9610-a2bd9b786fa7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:45.702397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e0f40-841a-49a6-bd77-61d33b730db6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:45.750208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d7d4-bd4f-4f52-ab98-7e8f9185c749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:45.751493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d7d4-bd4f-4f52-ab98-7e8f9185c749 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:45.752491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d7d4-bd4f-4f52-ab98-7e8f9185c749 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:45.753309 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38c302-9fd7-46c3-9045-2dd3d9aa6da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:45.754149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38c302-9fd7-46c3-9045-2dd3d9aa6da7 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:45.754638 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d7d4-bd4f-4f52-ab98-7e8f9185c749 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:45.755112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38c302-9fd7-46c3-9045-2dd3d9aa6da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:45.755846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38c302-9fd7-46c3-9045-2dd3d9aa6da7 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:45.756522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373d7d4-bd4f-4f52-ab98-7e8f9185c749 None admin] RBAC: Adding query filter params (name=network) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:45.757381 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38c302-9fd7-46c3-9045-2dd3d9aa6da7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:45.758483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373d7d4-bd4f-4f52-ab98-7e8f9185c749 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:45.760175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38c302-9fd7-46c3-9045-2dd3d9aa6da7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:45.764357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373d7d4-bd4f-4f52-ab98-7e8f9185c749 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:45.774196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 369/717] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:16:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:45.780404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d7d4-bd4f-4f52-ab98-7e8f9185c749 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:45.798464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 349/718] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:16:45 2022] POST /identity/v3/users => generated 254 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:16:45.889368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:45.949581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d7d4-bd4f-4f52-ab98-7e8f9185c749 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:45.999399 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597fb30-5325-4e48-80f6-9083228d7cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:46.000413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597fb30-5325-4e48-80f6-9083228d7cfb None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:46.000889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597fb30-5325-4e48-80f6-9083228d7cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:46.001353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597fb30-5325-4e48-80f6-9083228d7cfb None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:46.002186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597fb30-5325-4e48-80f6-9083228d7cfb None admin] RBAC: Adding query filter params (type=network) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:46.002719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597fb30-5325-4e48-80f6-9083228d7cfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:46.004318 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597fb30-5325-4e48-80f6-9083228d7cfb None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:46.013763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 370/719] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:16:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:46.734619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1aa6c-2f4a-4a69-b6cf-97260d7bcd16 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:46.735701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1aa6c-2f4a-4a69-b6cf-97260d7bcd16 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:46.736200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1aa6c-2f4a-4a69-b6cf-97260d7bcd16 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:46.738139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 350/720] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:46.743714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d60a0-8884-41f8-be33-700f9d6b7231 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:46.744540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d60a0-8884-41f8-be33-700f9d6b7231 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:46.745011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d60a0-8884-41f8-be33-700f9d6b7231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:46.768947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-518d60a0-8884-41f8-be33-700f9d6b7231 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:46.770444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-518d60a0-8884-41f8-be33-700f9d6b7231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:46.839398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 371/721] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:47.195611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1aa6c-2f4a-4a69-b6cf-97260d7bcd16 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:47.323688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1aa6c-2f4a-4a69-b6cf-97260d7bcd16 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:47.331311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:47.361860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c55d34-66fe-4cae-ace7-20ffe084b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:47.362587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c55d34-66fe-4cae-ace7-20ffe084b0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:47.362866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c55d34-66fe-4cae-ace7-20ffe084b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:47.363133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c55d34-66fe-4cae-ace7-20ffe084b0b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:47.375857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c55d34-66fe-4cae-ace7-20ffe084b0b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-97c55d34-66fe-4cae-ace7-20ffe084b0b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:47.380876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:47.381899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:47.382839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 351/722] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:47.389594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d60a0-8884-41f8-be33-700f9d6b7231 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:47.466935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:47.481370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f503-ecdc-4a85-b2ec-790309456288 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:47.481674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f503-ecdc-4a85-b2ec-790309456288 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:47.481915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f503-ecdc-4a85-b2ec-790309456288 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:47.483366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 352/723] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:16:47.488614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb979911-8cfd-4e6b-8426-87be0a17f823 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:47.488902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb979911-8cfd-4e6b-8426-87be0a17f823 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:47.489164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb979911-8cfd-4e6b-8426-87be0a17f823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:47.511264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d60a0-8884-41f8-be33-700f9d6b7231 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:47.525375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-eb979911-8cfd-4e6b-8426-87be0a17f823 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:47.526611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eb979911-8cfd-4e6b-8426-87be0a17f823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:47.557984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd5164-190d-404c-8706-dcda809c5f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:47.558600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd5164-190d-404c-8706-dcda809c5f5b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:47.558873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd5164-190d-404c-8706-dcda809c5f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:47.559139 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd5164-190d-404c-8706-dcda809c5f5b None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:47.559795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd5164-190d-404c-8706-dcda809c5f5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:47.560090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd5164-190d-404c-8706-dcda809c5f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:47.561464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd5164-190d-404c-8706-dcda809c5f5b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:47.573678 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 372/724] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:47.579192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd5164-190d-404c-8706-dcda809c5f5b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:47.663704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 353/725] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:47.723769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd5164-190d-404c-8706-dcda809c5f5b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:47.767065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:47.785097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624a487-ba7e-4c28-acbb-a276a82855ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:47.785725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624a487-ba7e-4c28-acbb-a276a82855ad None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:47.786000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624a487-ba7e-4c28-acbb-a276a82855ad None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:47.786256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624a487-ba7e-4c28-acbb-a276a82855ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:47.801793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624a487-ba7e-4c28-acbb-a276a82855ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:47.803290 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624a487-ba7e-4c28-acbb-a276a82855ad None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-8624a487-ba7e-4c28-acbb-a276a82855ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:47.819283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:47.820837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:47.821795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:47.821795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:47.821795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:47.821795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 373/726] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:47.827403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5234c9-7a18-47b0-b718-806606c56440 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:47.827859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5234c9-7a18-47b0-b718-806606c56440 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:47.827940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5234c9-7a18-47b0-b718-806606c56440 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:47.829242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624a487-ba7e-4c28-acbb-a276a82855ad None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:47.830691 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 354/727] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:47.834543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52650a-76d7-4153-874f-e4e4a72331ad None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:47.834825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52650a-76d7-4153-874f-e4e4a72331ad None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:47.835095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52650a-76d7-4153-874f-e4e4a72331ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:47.871603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1d52650a-76d7-4153-874f-e4e4a72331ad None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:47.872845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1d52650a-76d7-4153-874f-e4e4a72331ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:47.948526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624a487-ba7e-4c28-acbb-a276a82855ad None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:47.986603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 355/728] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:47.986762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff62405-a553-47e9-a43c-ed0252f0a966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:47.987323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff62405-a553-47e9-a43c-ed0252f0a966 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:47.987602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff62405-a553-47e9-a43c-ed0252f0a966 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:47.987873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff62405-a553-47e9-a43c-ed0252f0a966 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:47.988547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff62405-a553-47e9-a43c-ed0252f0a966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:47.988736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52650a-76d7-4153-874f-e4e4a72331ad None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:47.988849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff62405-a553-47e9-a43c-ed0252f0a966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:47.990196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff62405-a553-47e9-a43c-ed0252f0a966 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:48.009556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 374/729] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:48.015149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff62405-a553-47e9-a43c-ed0252f0a966 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:48.042462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:48.057767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:48.159574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff62405-a553-47e9-a43c-ed0252f0a966 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:48.195149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52650a-76d7-4153-874f-e4e4a72331ad None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:48.195948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b03c15-d3c6-4d3a-a87b-b669a7ce0d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:48.196609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b03c15-d3c6-4d3a-a87b-b669a7ce0d49 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:48.196883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b03c15-d3c6-4d3a-a87b-b669a7ce0d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:48.197152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b03c15-d3c6-4d3a-a87b-b669a7ce0d49 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:48.204525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b03c15-d3c6-4d3a-a87b-b669a7ce0d49 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:48.206784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b03c15-d3c6-4d3a-a87b-b669a7ce0d49 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-e9b03c15-d3c6-4d3a-a87b-b669a7ce0d49 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:48.219509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:48.220583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:48.221579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:48.221579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:48.221579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:48.221579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:48.221579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:48.221579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:48.221579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:48.221579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:48.221579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:48.221579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:48.225216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 375/730] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:48.227655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b03c15-d3c6-4d3a-a87b-b669a7ce0d49 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:48.301154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497fadf-2614-43cc-8c43-7e2f5d59e772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:48.302122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497fadf-2614-43cc-8c43-7e2f5d59e772 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:48.302621 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497fadf-2614-43cc-8c43-7e2f5d59e772 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:48.303093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497fadf-2614-43cc-8c43-7e2f5d59e772 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:48.303951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497fadf-2614-43cc-8c43-7e2f5d59e772 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:48.306086 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497fadf-2614-43cc-8c43-7e2f5d59e772 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:48.328410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:48.336423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b03c15-d3c6-4d3a-a87b-b669a7ce0d49 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:48.347778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 356/731] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:16:47 2022] POST /identity/v3/services => generated 251 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:16:48.395732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540b402-e1dc-49cd-a12c-650c52668a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:48.396145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497fadf-2614-43cc-8c43-7e2f5d59e772 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:48.402775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540b402-e1dc-49cd-a12c-650c52668a70 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:48.402775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540b402-e1dc-49cd-a12c-650c52668a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:48.402965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540b402-e1dc-49cd-a12c-650c52668a70 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:48.403748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540b402-e1dc-49cd-a12c-650c52668a70 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:48.404048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540b402-e1dc-49cd-a12c-650c52668a70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:48.405710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540b402-e1dc-49cd-a12c-650c52668a70 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:48.430691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 376/732] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 30 19:16:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:48.432023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540b402-e1dc-49cd-a12c-650c52668a70 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:48.542965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540b402-e1dc-49cd-a12c-650c52668a70 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:48.567762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:48.580331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b061a8-5c9f-4600-8fde-75daf0d46b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:48.580968 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b061a8-5c9f-4600-8fde-75daf0d46b38 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:48.581243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b061a8-5c9f-4600-8fde-75daf0d46b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:48.581505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b061a8-5c9f-4600-8fde-75daf0d46b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:48.586899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497fadf-2614-43cc-8c43-7e2f5d59e772 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:48.602899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:48.635655 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026742a-ed3a-41e8-bcf6-a8933c093c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:48.636701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026742a-ed3a-41e8-bcf6-a8933c093c8c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:48.637186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026742a-ed3a-41e8-bcf6-a8933c093c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:48.637671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026742a-ed3a-41e8-bcf6-a8933c093c8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:48.651028 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026742a-ed3a-41e8-bcf6-a8933c093c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9026742a-ed3a-41e8-bcf6-a8933c093c8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:48.656632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:48.657547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:48.658413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:48.661203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 357/733] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:48.667874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b061a8-5c9f-4600-8fde-75daf0d46b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:48.669775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b061a8-5c9f-4600-8fde-75daf0d46b38 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:48.670854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026742a-ed3a-41e8-bcf6-a8933c093c8c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:48.781972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 377/734] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:48 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 19:16:48.783805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31cf20-3608-4694-9490-d92e33f8ec27 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:48.784295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31cf20-3608-4694-9490-d92e33f8ec27 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:48.784732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31cf20-3608-4694-9490-d92e33f8ec27 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:48.786370 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 378/735] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:48.787898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31cf20-3608-4694-9490-d92e33f8ec27 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:48.817365 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026742a-ed3a-41e8-bcf6-a8933c093c8c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:48.844770 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:48.863509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5ea35-f113-408c-9ec9-6c394d6c3e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:48.864765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ea35-f113-408c-9ec9-6c394d6c3e7a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:48.865230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ea35-f113-408c-9ec9-6c394d6c3e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:48.865498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ea35-f113-408c-9ec9-6c394d6c3e7a None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:48.866369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5ea35-f113-408c-9ec9-6c394d6c3e7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:48.866859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5ea35-f113-408c-9ec9-6c394d6c3e7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:48.868433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5ea35-f113-408c-9ec9-6c394d6c3e7a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:48.878809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 358/736] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:48.880391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c60f5-dba3-4bdf-a1bc-da87115e46ca None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:48.880737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c60f5-dba3-4bdf-a1bc-da87115e46ca None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:48.881067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c60f5-dba3-4bdf-a1bc-da87115e46ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:48.901919 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31cf20-3608-4694-9490-d92e33f8ec27 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:48.903834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-919c60f5-dba3-4bdf-a1bc-da87115e46ca None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:48.905789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-919c60f5-dba3-4bdf-a1bc-da87115e46ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:48.907740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:48.938124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf3679-389f-4c42-a3d7-20f00eda7c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:48.938869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf3679-389f-4c42-a3d7-20f00eda7c31 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:48.939217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf3679-389f-4c42-a3d7-20f00eda7c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:48.939497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf3679-389f-4c42-a3d7-20f00eda7c31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:48.949708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf3679-389f-4c42-a3d7-20f00eda7c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:48.951307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf3679-389f-4c42-a3d7-20f00eda7c31 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-90cf3679-389f-4c42-a3d7-20f00eda7c31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:48.966570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:48.967691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:48.968769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:48.968769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:48.968769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:48.974529 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 379/737] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:48.975908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087f97c-8a81-48fc-831a-bea9d45475e7 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:48.976900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087f97c-8a81-48fc-831a-bea9d45475e7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:48.977315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087f97c-8a81-48fc-831a-bea9d45475e7 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:48.978708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 380/738] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:48.988417 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 359/739] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:48.988417 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c60f5-dba3-4bdf-a1bc-da87115e46ca None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:48.988631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087f97c-8a81-48fc-831a-bea9d45475e7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:49.103877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c60f5-dba3-4bdf-a1bc-da87115e46ca None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:49.104850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:49.110312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087f97c-8a81-48fc-831a-bea9d45475e7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:49.143524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db8837-ac34-493a-ab17-15787a4ca105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:49.144585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db8837-ac34-493a-ab17-15787a4ca105 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:49.145167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db8837-ac34-493a-ab17-15787a4ca105 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:49.145618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db8837-ac34-493a-ab17-15787a4ca105 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:49.146451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db8837-ac34-493a-ab17-15787a4ca105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:49.146930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db8837-ac34-493a-ab17-15787a4ca105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:49.148789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db8837-ac34-493a-ab17-15787a4ca105 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:49.153823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1476fd-5313-4b38-86a1-879847e09c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:49.154546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1476fd-5313-4b38-86a1-879847e09c98 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:49.154897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1476fd-5313-4b38-86a1-879847e09c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:49.155243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1476fd-5313-4b38-86a1-879847e09c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:49.157048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:49.164315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 360/740] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:49.165739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f52d4-aba7-4886-93ab-96f67f0937c0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:49.166276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f52d4-aba7-4886-93ab-96f67f0937c0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:49.166829 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f52d4-aba7-4886-93ab-96f67f0937c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:49.170799 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1476fd-5313-4b38-86a1-879847e09c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:49.172474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1476fd-5313-4b38-86a1-879847e09c98 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:49.186870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5c1f52d4-aba7-4886-93ab-96f67f0937c0 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:49.188473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5c1f52d4-aba7-4886-93ab-96f67f0937c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-0f1476fd-5313-4b38-86a1-879847e09c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:49.189343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:49.190276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:49.191228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:49.191228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:49.191228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:49.195246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 381/741] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:49.196416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1476fd-5313-4b38-86a1-879847e09c98 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:49.268355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 361/742] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 19:16:49.269748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f52d4-aba7-4886-93ab-96f67f0937c0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:49.322404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1476fd-5313-4b38-86a1-879847e09c98 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:49.358442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173a879-9973-448b-b959-7f3c262f50be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:49.359652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173a879-9973-448b-b959-7f3c262f50be None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:49.360113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173a879-9973-448b-b959-7f3c262f50be None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:49.360808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173a879-9973-448b-b959-7f3c262f50be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:49.371558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173a879-9973-448b-b959-7f3c262f50be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:49.374221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173a879-9973-448b-b959-7f3c262f50be None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c173a879-9973-448b-b959-7f3c262f50be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:49.392017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:49.393007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:49.393931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:49.393931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:49.393931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:49.393931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:49.393931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:49.393931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:49.393931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:49.393931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:49.396830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 382/743] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:49.397893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173a879-9973-448b-b959-7f3c262f50be None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:49.399031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f52d4-aba7-4886-93ab-96f67f0937c0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:49.416436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:49.435701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:49.447371 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bab8bd-fbdb-44ff-b4bc-2fd2d41e7c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:49.448154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bab8bd-fbdb-44ff-b4bc-2fd2d41e7c44 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:49.448595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bab8bd-fbdb-44ff-b4bc-2fd2d41e7c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:49.448971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bab8bd-fbdb-44ff-b4bc-2fd2d41e7c44 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:49.449748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bab8bd-fbdb-44ff-b4bc-2fd2d41e7c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:49.450175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bab8bd-fbdb-44ff-b4bc-2fd2d41e7c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:49.452576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bab8bd-fbdb-44ff-b4bc-2fd2d41e7c44 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:49.474423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 362/744] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:49.474423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bab8bd-fbdb-44ff-b4bc-2fd2d41e7c44 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:49.526349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173a879-9973-448b-b959-7f3c262f50be None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:49.564879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae05ac-f30f-4660-8b4f-c2a4e9cdd074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:49.565484 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae05ac-f30f-4660-8b4f-c2a4e9cdd074 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:49.565743 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae05ac-f30f-4660-8b4f-c2a4e9cdd074 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:49.566007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae05ac-f30f-4660-8b4f-c2a4e9cdd074 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:49.566645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae05ac-f30f-4660-8b4f-c2a4e9cdd074 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:49.567987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae05ac-f30f-4660-8b4f-c2a4e9cdd074 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:49.585034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 383/745] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:16:49 2022] POST /identity/v3/services => generated 240 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:16:49.586860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae05ac-f30f-4660-8b4f-c2a4e9cdd074 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:49.614712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bab8bd-fbdb-44ff-b4bc-2fd2d41e7c44 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:49.661056 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9d1f1-8012-4266-868e-e26db50fe905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:49.661781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d1f1-8012-4266-868e-e26db50fe905 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:49.662053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d1f1-8012-4266-868e-e26db50fe905 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:49.662318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d1f1-8012-4266-868e-e26db50fe905 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:49.663261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9d1f1-8012-4266-868e-e26db50fe905 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:49.663558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9d1f1-8012-4266-868e-e26db50fe905 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:49.665111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9d1f1-8012-4266-868e-e26db50fe905 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:49.684409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 363/746] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:49.685454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9d1f1-8012-4266-868e-e26db50fe905 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:49.697193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:49.740224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:49.754505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae05ac-f30f-4660-8b4f-c2a4e9cdd074 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:49.802098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb1b12-1e99-4831-944b-8226aa2a55ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:49.803403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb1b12-1e99-4831-944b-8226aa2a55ba None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:49.803837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb1b12-1e99-4831-944b-8226aa2a55ba None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:49.804113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb1b12-1e99-4831-944b-8226aa2a55ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:49.809980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9d1f1-8012-4266-868e-e26db50fe905 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:49.812728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb1b12-1e99-4831-944b-8226aa2a55ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:49.814178 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb1b12-1e99-4831-944b-8226aa2a55ba None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-88eb1b12-1e99-4831-944b-8226aa2a55ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:49.829189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:49.830182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 384/747] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 30 19:16:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:49.831164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb1b12-1e99-4831-944b-8226aa2a55ba None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:49.856488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f210fd-c3be-4edc-93c9-1f4cb499adc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:49.857406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f210fd-c3be-4edc-93c9-1f4cb499adc1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:49.857857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f210fd-c3be-4edc-93c9-1f4cb499adc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:49.858296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f210fd-c3be-4edc-93c9-1f4cb499adc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:49.869432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f210fd-c3be-4edc-93c9-1f4cb499adc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-33f210fd-c3be-4edc-93c9-1f4cb499adc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:49.876531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:49.878569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:49.880543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:49.889138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 364/748] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:49.890149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f210fd-c3be-4edc-93c9-1f4cb499adc1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:49.951883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb1b12-1e99-4831-944b-8226aa2a55ba None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:49.990663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:49.997800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbf046-b206-4f38-ac19-0dcb3e3b7df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:49.998497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbf046-b206-4f38-ac19-0dcb3e3b7df0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:49.998847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbf046-b206-4f38-ac19-0dcb3e3b7df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:49.999193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbf046-b206-4f38-ac19-0dcb3e3b7df0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:50.000440 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbf046-b206-4f38-ac19-0dcb3e3b7df0 None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=b8785c8c5ac84393afcaf88e891f3b7a) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:50.000820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbf046-b206-4f38-ac19-0dcb3e3b7df0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:50.002720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbf046-b206-4f38-ac19-0dcb3e3b7df0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:50.010570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 385/749] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:49 2022] GET /identity/v3/role_assignments?user.id=b8785c8c5ac84393afcaf88e891f3b7a&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:50.011988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbf046-b206-4f38-ac19-0dcb3e3b7df0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:50.024898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f210fd-c3be-4edc-93c9-1f4cb499adc1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:50.031957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:50.102665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075987c0-e2d2-4d47-89d7-f4e61c30a009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:50.103665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075987c0-e2d2-4d47-89d7-f4e61c30a009 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:50.104127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075987c0-e2d2-4d47-89d7-f4e61c30a009 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:50.104581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075987c0-e2d2-4d47-89d7-f4e61c30a009 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:50.105455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075987c0-e2d2-4d47-89d7-f4e61c30a009 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:50.105902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075987c0-e2d2-4d47-89d7-f4e61c30a009 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:50.107569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075987c0-e2d2-4d47-89d7-f4e61c30a009 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:50.139744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbf046-b206-4f38-ac19-0dcb3e3b7df0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:50.144030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 365/750] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 30 19:16:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:50.154125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075987c0-e2d2-4d47-89d7-f4e61c30a009 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:50.187918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bb620-a7a6-47ff-8a64-01916de5b32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:50.188786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bb620-a7a6-47ff-8a64-01916de5b32f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:50.190950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bb620-a7a6-47ff-8a64-01916de5b32f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:50.190950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bb620-a7a6-47ff-8a64-01916de5b32f None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:50.190950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bb620-a7a6-47ff-8a64-01916de5b32f None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:50.190950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bb620-a7a6-47ff-8a64-01916de5b32f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:50.192701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bb620-a7a6-47ff-8a64-01916de5b32f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:50.203112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 386/751] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:50.208527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bb620-a7a6-47ff-8a64-01916de5b32f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:50.281847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:50.343431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bb620-a7a6-47ff-8a64-01916de5b32f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:50.370369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075987c0-e2d2-4d47-89d7-f4e61c30a009 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:50.386516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594a3f3-9792-4448-b8e4-d78cdd58e554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:50.386516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594a3f3-9792-4448-b8e4-d78cdd58e554 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:50.386516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594a3f3-9792-4448-b8e4-d78cdd58e554 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:50.386516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594a3f3-9792-4448-b8e4-d78cdd58e554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:50.402019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594a3f3-9792-4448-b8e4-d78cdd58e554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:50.403869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594a3f3-9792-4448-b8e4-d78cdd58e554 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a594a3f3-9792-4448-b8e4-d78cdd58e554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:50.423683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:50.424760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:50.425697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:50.425697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:50.425697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:50.425697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 387/752] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:50.428518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15aabb-1fc3-4757-a2d2-7eb1f18b5058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:50.429962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15aabb-1fc3-4757-a2d2-7eb1f18b5058 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:50.430420 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15aabb-1fc3-4757-a2d2-7eb1f18b5058 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:50.430860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15aabb-1fc3-4757-a2d2-7eb1f18b5058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:50.430963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594a3f3-9792-4448-b8e4-d78cdd58e554 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:50.442673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15aabb-1fc3-4757-a2d2-7eb1f18b5058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:50.444470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15aabb-1fc3-4757-a2d2-7eb1f18b5058 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4a15aabb-1fc3-4757-a2d2-7eb1f18b5058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:50.461962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:50.462980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:50.463895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:50.463895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:50.463895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:50.466081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 366/753] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:50.474769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15aabb-1fc3-4757-a2d2-7eb1f18b5058 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:50.508879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:50.556928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594a3f3-9792-4448-b8e4-d78cdd58e554 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:50.588799 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:50.598463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18ba9c-b45b-4c4c-ba27-b0d3d5063c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:50.599544 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ba9c-b45b-4c4c-ba27-b0d3d5063c6e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:50.600152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ba9c-b45b-4c4c-ba27-b0d3d5063c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:50.600737 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ba9c-b45b-4c4c-ba27-b0d3d5063c6e None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:50.602090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18ba9c-b45b-4c4c-ba27-b0d3d5063c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:50.602999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18ba9c-b45b-4c4c-ba27-b0d3d5063c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:50.604564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18ba9c-b45b-4c4c-ba27-b0d3d5063c6e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:50.619711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 388/754] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:50.627756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18ba9c-b45b-4c4c-ba27-b0d3d5063c6e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:50.670293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15aabb-1fc3-4757-a2d2-7eb1f18b5058 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:50.717715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9815d-953e-472d-96f9-0ef3c81d8d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:50.718341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9815d-953e-472d-96f9-0ef3c81d8d1b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:50.718617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9815d-953e-472d-96f9-0ef3c81d8d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:50.718883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9815d-953e-472d-96f9-0ef3c81d8d1b None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:50.719528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9815d-953e-472d-96f9-0ef3c81d8d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:50.719826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9815d-953e-472d-96f9-0ef3c81d8d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:50.721213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9815d-953e-472d-96f9-0ef3c81d8d1b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:50.751951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 367/755] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:50.757584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9815d-953e-472d-96f9-0ef3c81d8d1b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:50.765256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18ba9c-b45b-4c4c-ba27-b0d3d5063c6e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:50.800845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5354961-ab53-42b4-95d1-da9c7e9a962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:50.801789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5354961-ab53-42b4-95d1-da9c7e9a962d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:50.802267 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5354961-ab53-42b4-95d1-da9c7e9a962d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:50.803167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5354961-ab53-42b4-95d1-da9c7e9a962d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:50.810993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5354961-ab53-42b4-95d1-da9c7e9a962d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:50.812744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5354961-ab53-42b4-95d1-da9c7e9a962d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d5354961-ab53-42b4-95d1-da9c7e9a962d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:50.825527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:50.826740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:50.828058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:50.828058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:50.828058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:50.828058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:50.828058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:50.828058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:50.828058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:50.828058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:50.828058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:16:50.828058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:50.830241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 389/756] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 30 19:16:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:50.835981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5354961-ab53-42b4-95d1-da9c7e9a962d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:50.861004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:50.880754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:50.888379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9815d-953e-472d-96f9-0ef3c81d8d1b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:50.923068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4504fc-ffb6-4545-a0c5-a8dea4f4051e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:50.923620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4504fc-ffb6-4545-a0c5-a8dea4f4051e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:50.923898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4504fc-ffb6-4545-a0c5-a8dea4f4051e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:50.924163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4504fc-ffb6-4545-a0c5-a8dea4f4051e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:50.931230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4504fc-ffb6-4545-a0c5-a8dea4f4051e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:50.932295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4504fc-ffb6-4545-a0c5-a8dea4f4051e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3e4504fc-ffb6-4545-a0c5-a8dea4f4051e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:50.943289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:50.944244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:50.945789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:50.945789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:50.945789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:50.945789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:50.945789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:50.945789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:50.945789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:50.945789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:50.946870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5354961-ab53-42b4-95d1-da9c7e9a962d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:50.950117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 368/757] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:50.950810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c5ead-8521-4b0d-827a-7ee5d50a83c0 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:50.951091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c5ead-8521-4b0d-827a-7ee5d50a83c0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:50.951363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c5ead-8521-4b0d-827a-7ee5d50a83c0 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:50.953377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 369/758] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:50.954231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c5ead-8521-4b0d-827a-7ee5d50a83c0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:50.990383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bbd86-ffa9-4faf-9be4-47556f6e9b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:50.991479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bbd86-ffa9-4faf-9be4-47556f6e9b4a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:50.992628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bbd86-ffa9-4faf-9be4-47556f6e9b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:50.993231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bbd86-ffa9-4faf-9be4-47556f6e9b4a None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:50.994160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bbd86-ffa9-4faf-9be4-47556f6e9b4a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:50.994669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bbd86-ffa9-4faf-9be4-47556f6e9b4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:50.996424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bbd86-ffa9-4faf-9be4-47556f6e9b4a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:51.021006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 390/759] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 30 19:16:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:51.022711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0e3b8-b4a9-44a7-b4f0-43225a58aea5 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:51.023292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0e3b8-b4a9-44a7-b4f0-43225a58aea5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:51.023772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0e3b8-b4a9-44a7-b4f0-43225a58aea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:51.043189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-69e0e3b8-b4a9-44a7-b4f0-43225a58aea5 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:51.044785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-69e0e3b8-b4a9-44a7-b4f0-43225a58aea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:51.077244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c5ead-8521-4b0d-827a-7ee5d50a83c0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:51.119102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3d44b-208e-4020-be41-2779d1c2d674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:51.119708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3d44b-208e-4020-be41-2779d1c2d674 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:51.119982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3d44b-208e-4020-be41-2779d1c2d674 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:51.120248 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3d44b-208e-4020-be41-2779d1c2d674 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:51.120994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3d44b-208e-4020-be41-2779d1c2d674 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:51.121279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3d44b-208e-4020-be41-2779d1c2d674 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:51.122910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3d44b-208e-4020-be41-2779d1c2d674 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:51.124596 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:51.125059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 391/760] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:16:51.126471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0e3b8-b4a9-44a7-b4f0-43225a58aea5 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:51.140461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 370/761] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:51.145097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3d44b-208e-4020-be41-2779d1c2d674 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:51.152967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:51.411572 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3d44b-208e-4020-be41-2779d1c2d674 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:51.444063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0e3b8-b4a9-44a7-b4f0-43225a58aea5 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:51.451501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c296d7-c2f3-4d73-af24-9006d32be233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:51.452153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c296d7-c2f3-4d73-af24-9006d32be233 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:51.452444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c296d7-c2f3-4d73-af24-9006d32be233 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:51.452759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c296d7-c2f3-4d73-af24-9006d32be233 None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/5952db29e6fc4ddbb9aeb0522497312c/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:51.454625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c296d7-c2f3-4d73-af24-9006d32be233 None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=5952db29e6fc4ddbb9aeb0522497312c, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:51.456044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c296d7-c2f3-4d73-af24-9006d32be233 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:51.475810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 371/762] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:16:51 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/5952db29e6fc4ddbb9aeb0522497312c/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:51.477318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c296d7-c2f3-4d73-af24-9006d32be233 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:51.498193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d407ef-ba7b-4629-bb34-492f2d39404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:51.498820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d407ef-ba7b-4629-bb34-492f2d39404d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:51.499106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d407ef-ba7b-4629-bb34-492f2d39404d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:51.499354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d407ef-ba7b-4629-bb34-492f2d39404d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:51.596574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:51.617386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d407ef-ba7b-4629-bb34-492f2d39404d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:51.618857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d407ef-ba7b-4629-bb34-492f2d39404d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:51.638243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:51.686695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c296d7-c2f3-4d73-af24-9006d32be233 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:51.729216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 392/763] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:16:51.737505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d407ef-ba7b-4629-bb34-492f2d39404d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:51.752613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2221c-b7ef-4a72-b6a9-387e0d21dd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:51.753335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2221c-b7ef-4a72-b6a9-387e0d21dd00 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:51.753610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2221c-b7ef-4a72-b6a9-387e0d21dd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:51.753879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2221c-b7ef-4a72-b6a9-387e0d21dd00 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:51.754472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2221c-b7ef-4a72-b6a9-387e0d21dd00 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:51.755997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2221c-b7ef-4a72-b6a9-387e0d21dd00 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5dd2221c-b7ef-4a72-b6a9-387e0d21dd00 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:51.772964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:51.774176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 19:16:51.775101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:51.775880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 372/764] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Sep 30 19:16:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:51.782443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2221c-b7ef-4a72-b6a9-387e0d21dd00 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:51.889220 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d407ef-ba7b-4629-bb34-492f2d39404d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:51.908328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:51.927428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24126abc-ed46-4caf-802f-5f471062c412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:51.928476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24126abc-ed46-4caf-802f-5f471062c412 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:51.928907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24126abc-ed46-4caf-802f-5f471062c412 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:51.929342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24126abc-ed46-4caf-802f-5f471062c412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:51.941559 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24126abc-ed46-4caf-802f-5f471062c412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:51.943545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24126abc-ed46-4caf-802f-5f471062c412 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:51.945900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2221c-b7ef-4a72-b6a9-387e0d21dd00 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-24126abc-ed46-4caf-802f-5f471062c412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:51.959780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:51.961340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:51.962255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:51.962255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:51.962255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:51.964078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 393/765] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:51.971718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24126abc-ed46-4caf-802f-5f471062c412 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:51.982930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:51.998151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6cc69-ce6e-4f83-b9c9-88a0ebaee39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:51.998910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6cc69-ce6e-4f83-b9c9-88a0ebaee39e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:51.999292 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6cc69-ce6e-4f83-b9c9-88a0ebaee39e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:51.999725 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6cc69-ce6e-4f83-b9c9-88a0ebaee39e None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.000489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6cc69-ce6e-4f83-b9c9-88a0ebaee39e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:52.000899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6cc69-ce6e-4f83-b9c9-88a0ebaee39e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:52.002503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6cc69-ce6e-4f83-b9c9-88a0ebaee39e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:52.008301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 373/766] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 30 19:16:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:52.013496 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6cc69-ce6e-4f83-b9c9-88a0ebaee39e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:52.182138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6cc69-ce6e-4f83-b9c9-88a0ebaee39e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:52.187926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24126abc-ed46-4caf-802f-5f471062c412 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:52.231363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36eae27-8965-445f-8226-f2f8f48d4a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:52.231881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36eae27-8965-445f-8226-f2f8f48d4a90 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.232152 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36eae27-8965-445f-8226-f2f8f48d4a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.232441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36eae27-8965-445f-8226-f2f8f48d4a90 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.233068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36eae27-8965-445f-8226-f2f8f48d4a90 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:52.233359 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36eae27-8965-445f-8226-f2f8f48d4a90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:52.234712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36eae27-8965-445f-8226-f2f8f48d4a90 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:52.236509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0f996-c9e3-4d83-beba-09d687beea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:52.237122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0f996-c9e3-4d83-beba-09d687beea29 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.237424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0f996-c9e3-4d83-beba-09d687beea29 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.237767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0f996-c9e3-4d83-beba-09d687beea29 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.238408 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0f996-c9e3-4d83-beba-09d687beea29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:52.238708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0f996-c9e3-4d83-beba-09d687beea29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:52.240064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0f996-c9e3-4d83-beba-09d687beea29 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:52.245440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 374/767] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 30 19:16:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:52.246753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43728c6-3659-499a-ab12-563d9d242b3c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.247033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43728c6-3659-499a-ab12-563d9d242b3c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.247302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43728c6-3659-499a-ab12-563d9d242b3c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.248868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 375/768] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:52.249589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43728c6-3659-499a-ab12-563d9d242b3c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:52.259341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 394/769] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:52.265422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64aea3-b146-45e6-afaa-9777f958169c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.265728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64aea3-b146-45e6-afaa-9777f958169c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.266012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64aea3-b146-45e6-afaa-9777f958169c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.291216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ec64aea3-b146-45e6-afaa-9777f958169c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:52.292647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ec64aea3-b146-45e6-afaa-9777f958169c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:52.309098 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:52.324939 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:52.373448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43728c6-3659-499a-ab12-563d9d242b3c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:52.377618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 395/770] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:16:52.378479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64aea3-b146-45e6-afaa-9777f958169c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:52.463958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b836a-cdcc-4a0d-948c-d0011cf56e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:52.464598 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b836a-cdcc-4a0d-948c-d0011cf56e86 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.464872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b836a-cdcc-4a0d-948c-d0011cf56e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.465136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b836a-cdcc-4a0d-948c-d0011cf56e86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.465841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b836a-cdcc-4a0d-948c-d0011cf56e86 None admin] RBAC: Adding query filter params (interface=public, service_id=e88bd0c51e52477fb776ab0a047f5591, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:52.466138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b836a-cdcc-4a0d-948c-d0011cf56e86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:52.467492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b836a-cdcc-4a0d-948c-d0011cf56e86 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:52.486554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 376/771] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 30 19:16:52 2022] GET /identity/v3/endpoints?service_id=e88bd0c51e52477fb776ab0a047f5591&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:52.489787 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64aea3-b146-45e6-afaa-9777f958169c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:52.492634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b836a-cdcc-4a0d-948c-d0011cf56e86 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:52.527614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bdee9-a8d5-48b2-b42b-ad05529aebbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:52.528299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bdee9-a8d5-48b2-b42b-ad05529aebbe None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.528644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bdee9-a8d5-48b2-b42b-ad05529aebbe None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.528913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bdee9-a8d5-48b2-b42b-ad05529aebbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.536038 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bdee9-a8d5-48b2-b42b-ad05529aebbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:52.537826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bdee9-a8d5-48b2-b42b-ad05529aebbe None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-e59bdee9-a8d5-48b2-b42b-ad05529aebbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:52.550268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:52.551206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:52.553185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:52.553185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:52.553185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:52.553185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:52.553185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:52.553185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:52.553185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:52.553185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:52.553185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 396/772] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:52.557485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bdee9-a8d5-48b2-b42b-ad05529aebbe None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:52.605085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:52.691784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bdee9-a8d5-48b2-b42b-ad05529aebbe None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:52.727433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b836a-cdcc-4a0d-948c-d0011cf56e86 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:52.728660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68950cb3-b4db-4c77-8e26-479b1a953b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:52.729268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68950cb3-b4db-4c77-8e26-479b1a953b97 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.729546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68950cb3-b4db-4c77-8e26-479b1a953b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.729818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68950cb3-b4db-4c77-8e26-479b1a953b97 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.730635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68950cb3-b4db-4c77-8e26-479b1a953b97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:52.730937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68950cb3-b4db-4c77-8e26-479b1a953b97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:52.740457 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68950cb3-b4db-4c77-8e26-479b1a953b97 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:52.744592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:52.746646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 397/773] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:52.747670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68950cb3-b4db-4c77-8e26-479b1a953b97 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:52.770490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fbbe6-1f33-4778-9b78-38c117278e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:52.771283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fbbe6-1f33-4778-9b78-38c117278e62 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.771626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fbbe6-1f33-4778-9b78-38c117278e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.771959 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fbbe6-1f33-4778-9b78-38c117278e62 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.772650 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fbbe6-1f33-4778-9b78-38c117278e62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:52.774718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fbbe6-1f33-4778-9b78-38c117278e62 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:52.784537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 377/774] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 30 19:16:52 2022] GET /identity/v3/services => generated 772 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:52.785605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fbbe6-1f33-4778-9b78-38c117278e62 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:52.839822 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:52.865108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68950cb3-b4db-4c77-8e26-479b1a953b97 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:52.905004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db7e3d-bda1-46b5-b5fc-e81166d6306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:52.905681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db7e3d-bda1-46b5-b5fc-e81166d6306c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.906022 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db7e3d-bda1-46b5-b5fc-e81166d6306c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.906302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db7e3d-bda1-46b5-b5fc-e81166d6306c None admin] PATH_INFO: `/v3/services/network` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.906897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db7e3d-bda1-46b5-b5fc-e81166d6306c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:52.908361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db7e3d-bda1-46b5-b5fc-e81166d6306c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d4db7e3d-bda1-46b5-b5fc-e81166d6306c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:52.918846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:52.919924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:52.921449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 398/775] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:52 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:52.922270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df24ad-a658-471a-a4ff-e4277dec3dfa None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.922270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df24ad-a658-471a-a4ff-e4277dec3dfa None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.922270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df24ad-a658-471a-a4ff-e4277dec3dfa None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.923533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 399/776] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:52.926594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fbbe6-1f33-4778-9b78-38c117278e62 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:52.928588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df24ad-a658-471a-a4ff-e4277dec3dfa None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:52.968507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58624693-b09b-40b7-83ee-09089d3ffb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:52.969934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58624693-b09b-40b7-83ee-09089d3ffb6e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.971061 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58624693-b09b-40b7-83ee-09089d3ffb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.971535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58624693-b09b-40b7-83ee-09089d3ffb6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:52.972587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58624693-b09b-40b7-83ee-09089d3ffb6e None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=b638b8149ae149e3aad92ed51be92588) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:52.973210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58624693-b09b-40b7-83ee-09089d3ffb6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:52.974892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58624693-b09b-40b7-83ee-09089d3ffb6e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:52.985087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 378/777] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:52 2022] GET /identity/v3/role_assignments?user.id=b638b8149ae149e3aad92ed51be92588&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:52.985702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e4b82-7c68-40a0-b16c-0803025897d5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:52.985983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e4b82-7c68-40a0-b16c-0803025897d5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:52.986252 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e4b82-7c68-40a0-b16c-0803025897d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:53.011415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ea4e4b82-7c68-40a0-b16c-0803025897d5 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:53.012580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ea4e4b82-7c68-40a0-b16c-0803025897d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:53.049762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:53.089208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df24ad-a658-471a-a4ff-e4277dec3dfa None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:53.093870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 379/778] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:16:53.137866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83346ce-acd9-4769-b789-ca82c2fe9b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:53.138497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83346ce-acd9-4769-b789-ca82c2fe9b6c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:53.138773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83346ce-acd9-4769-b789-ca82c2fe9b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:53.139041 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83346ce-acd9-4769-b789-ca82c2fe9b6c None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:53.139687 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83346ce-acd9-4769-b789-ca82c2fe9b6c None admin] RBAC: Adding query filter params (name=network) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:53.139985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83346ce-acd9-4769-b789-ca82c2fe9b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:53.141388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83346ce-acd9-4769-b789-ca82c2fe9b6c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:53.146867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 400/779] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:16:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:53.152273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e4b82-7c68-40a0-b16c-0803025897d5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:53.284265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e4b82-7c68-40a0-b16c-0803025897d5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:53.319438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21035798-6d90-43c5-9f50-33940d744075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:53.320123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21035798-6d90-43c5-9f50-33940d744075 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:53.320422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21035798-6d90-43c5-9f50-33940d744075 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:53.320693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21035798-6d90-43c5-9f50-33940d744075 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:53.321359 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21035798-6d90-43c5-9f50-33940d744075 None admin] RBAC: Adding query filter params (type=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:53.321656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21035798-6d90-43c5-9f50-33940d744075 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:53.323007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21035798-6d90-43c5-9f50-33940d744075 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:53.328193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 380/780] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:16:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:53.334002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83346ce-acd9-4769-b789-ca82c2fe9b6c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:53.346887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:53.443411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21035798-6d90-43c5-9f50-33940d744075 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:53.476224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83346ce-acd9-4769-b789-ca82c2fe9b6c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:53.494990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:53.512048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095114ce-f54b-4780-8310-0ce010550893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:53.512828 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095114ce-f54b-4780-8310-0ce010550893 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:53.513009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095114ce-f54b-4780-8310-0ce010550893 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:53.513235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095114ce-f54b-4780-8310-0ce010550893 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:53.513927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095114ce-f54b-4780-8310-0ce010550893 None admin] RBAC: Adding query filter params (interface=public, service_id=bd062c53f6c1432c86368619186f7cc5, region_id=RegionOne) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:53.514237 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095114ce-f54b-4780-8310-0ce010550893 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:53.515707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095114ce-f54b-4780-8310-0ce010550893 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:53.524085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 401/781] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 30 19:16:53 2022] GET /identity/v3/endpoints?service_id=bd062c53f6c1432c86368619186f7cc5&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:53.533005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095114ce-f54b-4780-8310-0ce010550893 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:53.581840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21035798-6d90-43c5-9f50-33940d744075 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:53.633009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0822fb-1e84-4f79-ba94-ea681a538bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:53.633830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0822fb-1e84-4f79-ba94-ea681a538bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:53.634291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0822fb-1e84-4f79-ba94-ea681a538bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:53.634654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0822fb-1e84-4f79-ba94-ea681a538bc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:53.640992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:53.660570 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0822fb-1e84-4f79-ba94-ea681a538bc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7e0822fb-1e84-4f79-ba94-ea681a538bc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:53.666753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:53.667696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:53.669020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 381/782] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:53.674537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0822fb-1e84-4f79-ba94-ea681a538bc8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:53.675863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095114ce-f54b-4780-8310-0ce010550893 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:53.742093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab7b78-8f9a-427b-9f27-06fab436d3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:53.742717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab7b78-8f9a-427b-9f27-06fab436d3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:53.742994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab7b78-8f9a-427b-9f27-06fab436d3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:53.743268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab7b78-8f9a-427b-9f27-06fab436d3f9 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:53.743873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab7b78-8f9a-427b-9f27-06fab436d3f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:53.745261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab7b78-8f9a-427b-9f27-06fab436d3f9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:53.759679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 402/783] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 30 19:16:53 2022] GET /identity/v3/services => generated 772 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:53.832186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:53.843672 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0822fb-1e84-4f79-ba94-ea681a538bc8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:53.889267 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b249007-ae6f-4c6d-919d-362b95aff34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:53.890253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b249007-ae6f-4c6d-919d-362b95aff34c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:53.890754 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b249007-ae6f-4c6d-919d-362b95aff34c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:53.891218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b249007-ae6f-4c6d-919d-362b95aff34c None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:53.892090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b249007-ae6f-4c6d-919d-362b95aff34c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:53.892618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b249007-ae6f-4c6d-919d-362b95aff34c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:53.894348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b249007-ae6f-4c6d-919d-362b95aff34c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:53.905508 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 382/784] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:53.910824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab7b78-8f9a-427b-9f27-06fab436d3f9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:54.054138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab7b78-8f9a-427b-9f27-06fab436d3f9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:54.089385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85458208-bcce-4dd7-8425-718b3abd1643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:54.089945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85458208-bcce-4dd7-8425-718b3abd1643 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:54.090222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85458208-bcce-4dd7-8425-718b3abd1643 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:54.090491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85458208-bcce-4dd7-8425-718b3abd1643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:54.094203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:54.096732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8586dc-a145-4316-9fcc-b6a901d901ec None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:54.097194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8586dc-a145-4316-9fcc-b6a901d901ec None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:54.097303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8586dc-a145-4316-9fcc-b6a901d901ec None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:54.099756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 383/785] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:54.105536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84c98d-1d02-4b22-b3d3-696de862e54e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:54.105815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84c98d-1d02-4b22-b3d3-696de862e54e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:54.106088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84c98d-1d02-4b22-b3d3-696de862e54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:54.108196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85458208-bcce-4dd7-8425-718b3abd1643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:54.109923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85458208-bcce-4dd7-8425-718b3abd1643 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-85458208-bcce-4dd7-8425-718b3abd1643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:54.125327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:54.126361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:54.127290 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:54.127290 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:54.127290 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:54.129309 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5d84c98d-1d02-4b22-b3d3-696de862e54e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:54.129554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 403/786] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:54.130474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5d84c98d-1d02-4b22-b3d3-696de862e54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:54.139127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85458208-bcce-4dd7-8425-718b3abd1643 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:54.268001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 384/787] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:16:54.305168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85458208-bcce-4dd7-8425-718b3abd1643 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:54.361790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8222c0-3905-4970-8ff0-53742573909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:54.362327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8222c0-3905-4970-8ff0-53742573909d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:54.362602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8222c0-3905-4970-8ff0-53742573909d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:54.362869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8222c0-3905-4970-8ff0-53742573909d None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:54.363506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8222c0-3905-4970-8ff0-53742573909d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:54.363805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8222c0-3905-4970-8ff0-53742573909d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:54.365194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8222c0-3905-4970-8ff0-53742573909d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:54.382293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 404/788] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:54.387583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84c98d-1d02-4b22-b3d3-696de862e54e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:54.494211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:54.511533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84c98d-1d02-4b22-b3d3-696de862e54e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:54.547705 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b50a2-1de7-4ab1-83e7-ca3e32891414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:54.548330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b50a2-1de7-4ab1-83e7-ca3e32891414 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:54.548635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b50a2-1de7-4ab1-83e7-ca3e32891414 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:54.548905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b50a2-1de7-4ab1-83e7-ca3e32891414 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:54.556096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b50a2-1de7-4ab1-83e7-ca3e32891414 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:54.557564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b50a2-1de7-4ab1-83e7-ca3e32891414 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6a0b50a2-1de7-4ab1-83e7-ca3e32891414 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:54.569330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:54.570875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:54.572238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:54.572238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:54.572238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:54.572238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:54.572238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:54.572238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:54.572238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:54.572238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:54.572238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:54.572238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:54.572238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 385/789] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:54.577434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8222c0-3905-4970-8ff0-53742573909d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:54.646004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:54.698160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8222c0-3905-4970-8ff0-53742573909d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:54.710184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b50a2-1de7-4ab1-83e7-ca3e32891414 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:54.746454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf4aa0-e431-48e4-a177-2f19382adfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:54.747069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf4aa0-e431-48e4-a177-2f19382adfed None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:54.747345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf4aa0-e431-48e4-a177-2f19382adfed None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:54.747609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf4aa0-e431-48e4-a177-2f19382adfed None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:54.748333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf4aa0-e431-48e4-a177-2f19382adfed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:54.748652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf4aa0-e431-48e4-a177-2f19382adfed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:54.750069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf4aa0-e431-48e4-a177-2f19382adfed None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:54.790389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 405/790] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 30 19:16:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:54.796654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf4aa0-e431-48e4-a177-2f19382adfed None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:54.859185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b50a2-1de7-4ab1-83e7-ca3e32891414 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:54.926757 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a45be-30ad-48a1-bbfa-b9fae6cd9b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:54.927753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a45be-30ad-48a1-bbfa-b9fae6cd9b8d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:54.928227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a45be-30ad-48a1-bbfa-b9fae6cd9b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:54.928713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a45be-30ad-48a1-bbfa-b9fae6cd9b8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:54.950351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a45be-30ad-48a1-bbfa-b9fae6cd9b8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-164a45be-30ad-48a1-bbfa-b9fae6cd9b8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:54.963369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:54.964434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:54.965304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:54.972821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 386/791] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:54.974105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a45be-30ad-48a1-bbfa-b9fae6cd9b8d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:54.992753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:55.002349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf4aa0-e431-48e4-a177-2f19382adfed None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:55.016705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:55.074213 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65ff39-180f-47be-99a7-58d4ab724360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:55.074839 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ff39-180f-47be-99a7-58d4ab724360 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.075111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ff39-180f-47be-99a7-58d4ab724360 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.075376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65ff39-180f-47be-99a7-58d4ab724360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.093744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65ff39-180f-47be-99a7-58d4ab724360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:55.095177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65ff39-180f-47be-99a7-58d4ab724360 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-9f65ff39-180f-47be-99a7-58d4ab724360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:55.121312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:55.122340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:55.123253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:55.123253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:55.123253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:55.123253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 406/792] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:55.128174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65ff39-180f-47be-99a7-58d4ab724360 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:55.167142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a45be-30ad-48a1-bbfa-b9fae6cd9b8d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:55.231503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422e494-38d2-4794-b1e8-6e924b563fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:55.232542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422e494-38d2-4794-b1e8-6e924b563fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.233190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422e494-38d2-4794-b1e8-6e924b563fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.234773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422e494-38d2-4794-b1e8-6e924b563fd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.235648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422e494-38d2-4794-b1e8-6e924b563fd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:55.236160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422e494-38d2-4794-b1e8-6e924b563fd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:55.238434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422e494-38d2-4794-b1e8-6e924b563fd7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:55.259196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 387/793] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:55.264478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422e494-38d2-4794-b1e8-6e924b563fd7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:55.307056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65ff39-180f-47be-99a7-58d4ab724360 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:55.374091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a36b0d-bd2e-43ec-a034-6a70dbc05144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:55.374702 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a36b0d-bd2e-43ec-a034-6a70dbc05144 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.374980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a36b0d-bd2e-43ec-a034-6a70dbc05144 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.375252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a36b0d-bd2e-43ec-a034-6a70dbc05144 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.375890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a36b0d-bd2e-43ec-a034-6a70dbc05144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:55.376193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a36b0d-bd2e-43ec-a034-6a70dbc05144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:55.377578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a36b0d-bd2e-43ec-a034-6a70dbc05144 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:55.379133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:55.392758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 407/794] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:16:55.405137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a36b0d-bd2e-43ec-a034-6a70dbc05144 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:55.443634 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:55.452280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422e494-38d2-4794-b1e8-6e924b563fd7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:55.498162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bde716-59db-426a-af2a-4cd42a3d9645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:55.499403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bde716-59db-426a-af2a-4cd42a3d9645 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.499901 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bde716-59db-426a-af2a-4cd42a3d9645 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.500851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bde716-59db-426a-af2a-4cd42a3d9645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.514647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bde716-59db-426a-af2a-4cd42a3d9645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:55.516613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bde716-59db-426a-af2a-4cd42a3d9645 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:55.535604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a36b0d-bd2e-43ec-a034-6a70dbc05144 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-90bde716-59db-426a-af2a-4cd42a3d9645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:55.536044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:55.537020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:55.538193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:55.538193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:55.538193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:55.541113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 388/795] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:16:55.545759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9db42-42dd-482c-a2d6-42a6a8fabb13 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.545759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9db42-42dd-482c-a2d6-42a6a8fabb13 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.545759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9db42-42dd-482c-a2d6-42a6a8fabb13 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.552490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 389/796] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:55.553256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9db42-42dd-482c-a2d6-42a6a8fabb13 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:55.578978 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e9817-3cf6-4ae5-9286-0a237f1d71e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:55.580098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e9817-3cf6-4ae5-9286-0a237f1d71e4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.580784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e9817-3cf6-4ae5-9286-0a237f1d71e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.581699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e9817-3cf6-4ae5-9286-0a237f1d71e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.591493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e9817-3cf6-4ae5-9286-0a237f1d71e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:55.593338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e9817-3cf6-4ae5-9286-0a237f1d71e4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ce0e9817-3cf6-4ae5-9286-0a237f1d71e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:55.605953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:55.606938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:55.607863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:55.607863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:55.607863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:55.607863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:55.607863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:55.607863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:55.607863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:55.607863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:55.610749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 408/797] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:55.615659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd96824-c469-4bd1-9dcf-2a2f3cf6c278 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.615659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd96824-c469-4bd1-9dcf-2a2f3cf6c278 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.615659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd96824-c469-4bd1-9dcf-2a2f3cf6c278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.640401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-afd96824-c469-4bd1-9dcf-2a2f3cf6c278 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:55.641575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-afd96824-c469-4bd1-9dcf-2a2f3cf6c278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:55.689932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9db42-42dd-482c-a2d6-42a6a8fabb13 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:55.710253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 409/798] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:16:55.711690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd96824-c469-4bd1-9dcf-2a2f3cf6c278 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:55.712989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:55.724459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:55.734637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965244bc-0b62-425f-8726-38144d0ea359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:55.735534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965244bc-0b62-425f-8726-38144d0ea359 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.736028 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965244bc-0b62-425f-8726-38144d0ea359 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.736549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965244bc-0b62-425f-8726-38144d0ea359 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.737523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965244bc-0b62-425f-8726-38144d0ea359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:55.738015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965244bc-0b62-425f-8726-38144d0ea359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:55.739701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965244bc-0b62-425f-8726-38144d0ea359 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:55.752990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 390/799] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:55.754669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dff630-6b46-4eb5-ae75-81650f864848 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.755145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dff630-6b46-4eb5-ae75-81650f864848 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.755602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dff630-6b46-4eb5-ae75-81650f864848 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.758032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 391/800] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:55.758933 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dff630-6b46-4eb5-ae75-81650f864848 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:55.839334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd96824-c469-4bd1-9dcf-2a2f3cf6c278 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:55.868546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dff630-6b46-4eb5-ae75-81650f864848 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:55.887751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7bcad-3cd9-41d3-9a65-42e0eaa42246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:55.888358 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7bcad-3cd9-41d3-9a65-42e0eaa42246 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.888823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7bcad-3cd9-41d3-9a65-42e0eaa42246 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.889095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7bcad-3cd9-41d3-9a65-42e0eaa42246 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.889773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7bcad-3cd9-41d3-9a65-42e0eaa42246 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:55.890075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7bcad-3cd9-41d3-9a65-42e0eaa42246 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:55.891489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7bcad-3cd9-41d3-9a65-42e0eaa42246 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:55.908576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 410/801] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:55.908576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b245db-13d6-4f81-8b69-657302ff37c0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.908576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b245db-13d6-4f81-8b69-657302ff37c0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.908576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b245db-13d6-4f81-8b69-657302ff37c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.919150 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dc0f7-2d74-4c1a-b47c-c70cd2315135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:55.920052 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dc0f7-2d74-4c1a-b47c-c70cd2315135 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:55.920571 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dc0f7-2d74-4c1a-b47c-c70cd2315135 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:55.921015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dc0f7-2d74-4c1a-b47c-c70cd2315135 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:55.927155 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c7b245db-13d6-4f81-8b69-657302ff37c0 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:55.928473 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c7b245db-13d6-4f81-8b69-657302ff37c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:55.929208 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15dc0f7-2d74-4c1a-b47c-c70cd2315135 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:55.930907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15dc0f7-2d74-4c1a-b47c-c70cd2315135 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a15dc0f7-2d74-4c1a-b47c-c70cd2315135 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:55.943113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:55.944106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:55.945797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:55.945797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:55.945797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:55.945797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:55.945797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:55.945797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:55.945797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:55.945797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:55.945797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:16:55.945797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:55.952527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 392/802] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 30 19:16:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:55.953559 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dc0f7-2d74-4c1a-b47c-c70cd2315135 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:56.000663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:56.004018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:56.010676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 411/803] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:16:56.011409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b245db-13d6-4f81-8b69-657302ff37c0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:56.071301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dc0f7-2d74-4c1a-b47c-c70cd2315135 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:56.107171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dad8fd-6f5c-4996-8254-2f8a4426bc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:56.108199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dad8fd-6f5c-4996-8254-2f8a4426bc60 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:56.108703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dad8fd-6f5c-4996-8254-2f8a4426bc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:56.109170 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dad8fd-6f5c-4996-8254-2f8a4426bc60 None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/b8785c8c5ac84393afcaf88e891f3b7a/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:56.111460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dad8fd-6f5c-4996-8254-2f8a4426bc60 None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=b8785c8c5ac84393afcaf88e891f3b7a, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:56.113216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dad8fd-6f5c-4996-8254-2f8a4426bc60 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:56.124133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b245db-13d6-4f81-8b69-657302ff37c0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:56.130598 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 393/804] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:16:55 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/b8785c8c5ac84393afcaf88e891f3b7a/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:16:56.137268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dad8fd-6f5c-4996-8254-2f8a4426bc60 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:56.198344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538b192-9c0a-4e26-b8fd-8907ab2f32f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:56.198978 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538b192-9c0a-4e26-b8fd-8907ab2f32f7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:56.199249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538b192-9c0a-4e26-b8fd-8907ab2f32f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:56.200514 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538b192-9c0a-4e26-b8fd-8907ab2f32f7 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:56.201310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538b192-9c0a-4e26-b8fd-8907ab2f32f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:56.201619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538b192-9c0a-4e26-b8fd-8907ab2f32f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:56.203072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538b192-9c0a-4e26-b8fd-8907ab2f32f7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:56.243979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 412/805] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 30 19:16:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:56.251432 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538b192-9c0a-4e26-b8fd-8907ab2f32f7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:56.283035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:56.292781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dad8fd-6f5c-4996-8254-2f8a4426bc60 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:56.341936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8274870-894e-4ea0-b997-47b0e9c83fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:56.342588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8274870-894e-4ea0-b997-47b0e9c83fce None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:56.342993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8274870-894e-4ea0-b997-47b0e9c83fce None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:56.343276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8274870-894e-4ea0-b997-47b0e9c83fce None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:56.343874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8274870-894e-4ea0-b997-47b0e9c83fce None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:56.346284 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8274870-894e-4ea0-b997-47b0e9c83fce None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:56.348506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-c8274870-894e-4ea0-b997-47b0e9c83fce None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:56.357662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:56.358746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:56.359637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 394/806] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Sep 30 19:16:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:56.360867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488eed2-0185-4746-b337-439cb4dff655 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:56.360867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488eed2-0185-4746-b337-439cb4dff655 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:56.360867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488eed2-0185-4746-b337-439cb4dff655 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:56.362053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 395/807] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:56.362785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488eed2-0185-4746-b337-439cb4dff655 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:56.416224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538b192-9c0a-4e26-b8fd-8907ab2f32f7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:56.453079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a1d8a-6c4b-48ec-9dbe-f730ab364d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:56.453696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a1d8a-6c4b-48ec-9dbe-f730ab364d1f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:56.453972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a1d8a-6c4b-48ec-9dbe-f730ab364d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:56.454245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a1d8a-6c4b-48ec-9dbe-f730ab364d1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:56.476503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488eed2-0185-4746-b337-439cb4dff655 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:56.510667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdb973-f023-4dfc-bfd3-96203c2bbcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:56.511291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdb973-f023-4dfc-bfd3-96203c2bbcf0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:56.511564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdb973-f023-4dfc-bfd3-96203c2bbcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:56.511823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdb973-f023-4dfc-bfd3-96203c2bbcf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:56.521786 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdb973-f023-4dfc-bfd3-96203c2bbcf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:56.527415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a1d8a-6c4b-48ec-9dbe-f730ab364d1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1acdb973-f023-4dfc-bfd3-96203c2bbcf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:56.527566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:56.528807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:16:56.529718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:56.530378 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a1d8a-6c4b-48ec-9dbe-f730ab364d1f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:56.530452 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 396/808] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:16:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:56.530452 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdb973-f023-4dfc-bfd3-96203c2bbcf0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:56.554328 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:56.594808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:56.630579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdb973-f023-4dfc-bfd3-96203c2bbcf0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:56.644147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 413/809] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:16:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 30 19:16:56.645630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5d394-150b-4b43-9d5f-b53dc66e6192 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:56.646144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5d394-150b-4b43-9d5f-b53dc66e6192 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:56.646574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5d394-150b-4b43-9d5f-b53dc66e6192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:56.666482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e4d5d394-150b-4b43-9d5f-b53dc66e6192 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:56.667650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e4d5d394-150b-4b43-9d5f-b53dc66e6192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:56.670768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed01e5b-d5a1-44fe-a24c-fb9ffedd92a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:56.671380 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed01e5b-d5a1-44fe-a24c-fb9ffedd92a6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:56.671658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed01e5b-d5a1-44fe-a24c-fb9ffedd92a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:56.671927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed01e5b-d5a1-44fe-a24c-fb9ffedd92a6 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:56.672575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed01e5b-d5a1-44fe-a24c-fb9ffedd92a6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:56.672875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed01e5b-d5a1-44fe-a24c-fb9ffedd92a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:56.674229 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed01e5b-d5a1-44fe-a24c-fb9ffedd92a6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:56.680414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 397/810] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 30 19:16:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:56.680414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed01e5b-d5a1-44fe-a24c-fb9ffedd92a6 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:56.752398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 414/811] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:16:56.752398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5d394-150b-4b43-9d5f-b53dc66e6192 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:56.781767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:56.800023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed01e5b-d5a1-44fe-a24c-fb9ffedd92a6 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:56.835062 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac102507-6af9-484d-9bb8-4cfdcb4f13a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:56.836293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac102507-6af9-484d-9bb8-4cfdcb4f13a1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:56.836602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac102507-6af9-484d-9bb8-4cfdcb4f13a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:56.836870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac102507-6af9-484d-9bb8-4cfdcb4f13a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:56.837531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac102507-6af9-484d-9bb8-4cfdcb4f13a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:56.837825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac102507-6af9-484d-9bb8-4cfdcb4f13a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:56.839190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac102507-6af9-484d-9bb8-4cfdcb4f13a1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:56.843249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:56.852933 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 398/812] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:16:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:56.853948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac102507-6af9-484d-9bb8-4cfdcb4f13a1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:56.859046 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5d394-150b-4b43-9d5f-b53dc66e6192 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:56.893924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7c59f-6a51-4d85-b2ff-5c16cacb05d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:56.894549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7c59f-6a51-4d85-b2ff-5c16cacb05d7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:56.894820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7c59f-6a51-4d85-b2ff-5c16cacb05d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:56.895081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7c59f-6a51-4d85-b2ff-5c16cacb05d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:56.906847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7c59f-6a51-4d85-b2ff-5c16cacb05d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:56.908386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7c59f-6a51-4d85-b2ff-5c16cacb05d7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-20d7c59f-6a51-4d85-b2ff-5c16cacb05d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:56.924950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:56.925972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:56.926901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:56.926901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:56.926901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:56.926901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 415/813] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:56.926901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7c59f-6a51-4d85-b2ff-5c16cacb05d7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:56.979046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac102507-6af9-484d-9bb8-4cfdcb4f13a1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:57.021488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a777e9e-ab26-406d-b1f3-1a393169ddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:57.022493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a777e9e-ab26-406d-b1f3-1a393169ddef None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:57.023331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a777e9e-ab26-406d-b1f3-1a393169ddef None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:57.023698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a777e9e-ab26-406d-b1f3-1a393169ddef None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:57.024705 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a777e9e-ab26-406d-b1f3-1a393169ddef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:57.025311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a777e9e-ab26-406d-b1f3-1a393169ddef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:57.027153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a777e9e-ab26-406d-b1f3-1a393169ddef None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:57.034232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 399/814] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Sep 30 19:16:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:57.035090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a777e9e-ab26-406d-b1f3-1a393169ddef None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:57.046339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7c59f-6a51-4d85-b2ff-5c16cacb05d7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:57.053407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:57.083575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8322f2-ab59-47bc-ab0a-467d17445496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:57.084675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8322f2-ab59-47bc-ab0a-467d17445496 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:57.084954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8322f2-ab59-47bc-ab0a-467d17445496 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:57.085277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8322f2-ab59-47bc-ab0a-467d17445496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:57.094076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:57.098820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8322f2-ab59-47bc-ab0a-467d17445496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:57.100817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8322f2-ab59-47bc-ab0a-467d17445496 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-df8322f2-ab59-47bc-ab0a-467d17445496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:57.116862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:57.117852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:57.118780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:57.118780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:57.118780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:57.118780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 416/815] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:57.118780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8322f2-ab59-47bc-ab0a-467d17445496 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:57.152535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a777e9e-ab26-406d-b1f3-1a393169ddef None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:57.187825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910d08a-9aa9-48a3-b1d3-55bee10dca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:57.188323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910d08a-9aa9-48a3-b1d3-55bee10dca52 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:57.188626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910d08a-9aa9-48a3-b1d3-55bee10dca52 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:57.188899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910d08a-9aa9-48a3-b1d3-55bee10dca52 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:57.189527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910d08a-9aa9-48a3-b1d3-55bee10dca52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:57.189826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910d08a-9aa9-48a3-b1d3-55bee10dca52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:57.191245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910d08a-9aa9-48a3-b1d3-55bee10dca52 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:57.203107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 400/816] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:57.203954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910d08a-9aa9-48a3-b1d3-55bee10dca52 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:57.229768 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8322f2-ab59-47bc-ab0a-467d17445496 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:57.264516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffedd4b-04b2-443f-939e-2e02bf8a4e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:57.265312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffedd4b-04b2-443f-939e-2e02bf8a4e3c None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:57.265588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffedd4b-04b2-443f-939e-2e02bf8a4e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:57.266181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffedd4b-04b2-443f-939e-2e02bf8a4e3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:57.266835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffedd4b-04b2-443f-939e-2e02bf8a4e3c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:57.268184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffedd4b-04b2-443f-939e-2e02bf8a4e3c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:57.275198 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:57.293008 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 417/817] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:16:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:16:57.293829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffedd4b-04b2-443f-939e-2e02bf8a4e3c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:57.320481 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910d08a-9aa9-48a3-b1d3-55bee10dca52 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:57.354728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:57.380408 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edc1cd-9b4f-4f3a-8881-bbaf86bdcda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:57.381452 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edc1cd-9b4f-4f3a-8881-bbaf86bdcda0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:57.382064 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edc1cd-9b4f-4f3a-8881-bbaf86bdcda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:57.383951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edc1cd-9b4f-4f3a-8881-bbaf86bdcda0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:57.384840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edc1cd-9b4f-4f3a-8881-bbaf86bdcda0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:57.386437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edc1cd-9b4f-4f3a-8881-bbaf86bdcda0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-15edc1cd-9b4f-4f3a-8881-bbaf86bdcda0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:57.399660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:57.401344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:57.402197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 401/818] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:57 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:57.403043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edc1cd-9b4f-4f3a-8881-bbaf86bdcda0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:57.434453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffedd4b-04b2-443f-939e-2e02bf8a4e3c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:57.484006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615be517-9cfd-4157-a297-20e1cd880adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:57.485154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615be517-9cfd-4157-a297-20e1cd880adf None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:57.485577 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615be517-9cfd-4157-a297-20e1cd880adf None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:57.486241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615be517-9cfd-4157-a297-20e1cd880adf None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:57.487911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615be517-9cfd-4157-a297-20e1cd880adf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:57.488352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615be517-9cfd-4157-a297-20e1cd880adf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:57.489900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615be517-9cfd-4157-a297-20e1cd880adf None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:57.509952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 418/819] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:57.512156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615be517-9cfd-4157-a297-20e1cd880adf None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:57.526330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edc1cd-9b4f-4f3a-8881-bbaf86bdcda0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:57.560187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:57.565363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90a65c-1e54-40c9-8e55-ae4fcd9ba6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:57.566386 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90a65c-1e54-40c9-8e55-ae4fcd9ba6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:57.567046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90a65c-1e54-40c9-8e55-ae4fcd9ba6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:57.567310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90a65c-1e54-40c9-8e55-ae4fcd9ba6a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:57.577623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90a65c-1e54-40c9-8e55-ae4fcd9ba6a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:57.579034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90a65c-1e54-40c9-8e55-ae4fcd9ba6a1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-8e90a65c-1e54-40c9-8e55-ae4fcd9ba6a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:57.590379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:57.591374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:57.592348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:57.592348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:57.592348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:57.592348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:57.592348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:57.592348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:57.592348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:57.592348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:57.592348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 402/820] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:57.592882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90a65c-1e54-40c9-8e55-ae4fcd9ba6a1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:57.647276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615be517-9cfd-4157-a297-20e1cd880adf None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:57.656227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:57.687581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad85483-1a74-4790-9ab4-d210df504bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:57.688643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad85483-1a74-4790-9ab4-d210df504bad None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:57.688923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad85483-1a74-4790-9ab4-d210df504bad None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:57.689191 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad85483-1a74-4790-9ab4-d210df504bad None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:57.690132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad85483-1a74-4790-9ab4-d210df504bad None admin] RBAC: Adding query filter params (name=network) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:57.690435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad85483-1a74-4790-9ab4-d210df504bad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:57.692060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad85483-1a74-4790-9ab4-d210df504bad None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:57.699464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 419/821] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:16:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:57.701532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad85483-1a74-4790-9ab4-d210df504bad None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:57.714058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90a65c-1e54-40c9-8e55-ae4fcd9ba6a1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:57.754662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77f0a6-bf57-4d2f-8ecc-69ec31321d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:57.755467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77f0a6-bf57-4d2f-8ecc-69ec31321d5f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:57.755844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77f0a6-bf57-4d2f-8ecc-69ec31321d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:57.756214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77f0a6-bf57-4d2f-8ecc-69ec31321d5f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:57.765176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77f0a6-bf57-4d2f-8ecc-69ec31321d5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:57.766758 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77f0a6-bf57-4d2f-8ecc-69ec31321d5f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4a77f0a6-bf57-4d2f-8ecc-69ec31321d5f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:57.778918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:57.780503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 403/822] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 30 19:16:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:57.782685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77f0a6-bf57-4d2f-8ecc-69ec31321d5f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:57.828029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad85483-1a74-4790-9ab4-d210df504bad None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:57.855470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:57.871673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0bf3b-9192-455d-8d4d-84efa2234eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:57.872289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0bf3b-9192-455d-8d4d-84efa2234eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:57.872611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0bf3b-9192-455d-8d4d-84efa2234eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:57.872883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0bf3b-9192-455d-8d4d-84efa2234eb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:57.873569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0bf3b-9192-455d-8d4d-84efa2234eb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:57.873870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0bf3b-9192-455d-8d4d-84efa2234eb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:57.875831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0bf3b-9192-455d-8d4d-84efa2234eb4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:57.892508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 420/823] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:57.893710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0bf3b-9192-455d-8d4d-84efa2234eb4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:57.919392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77f0a6-bf57-4d2f-8ecc-69ec31321d5f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:57.930214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:57.955972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfea60-78fa-4727-a576-120939f4aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:57.956611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfea60-78fa-4727-a576-120939f4aeaa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:57.956888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfea60-78fa-4727-a576-120939f4aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:57.957159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfea60-78fa-4727-a576-120939f4aeaa None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:57.957789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfea60-78fa-4727-a576-120939f4aeaa None admin] RBAC: Adding query filter params (type=network) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:57.958088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfea60-78fa-4727-a576-120939f4aeaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:57.960017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfea60-78fa-4727-a576-120939f4aeaa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:57.972907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 404/824] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:16:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:57.974238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfea60-78fa-4727-a576-120939f4aeaa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:58.019086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0bf3b-9192-455d-8d4d-84efa2234eb4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:58.064835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4276e61-3871-4641-8bbd-84ae13be1c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:58.066285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4276e61-3871-4641-8bbd-84ae13be1c6c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:58.066769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4276e61-3871-4641-8bbd-84ae13be1c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:58.067245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4276e61-3871-4641-8bbd-84ae13be1c6c None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:58.068213 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4276e61-3871-4641-8bbd-84ae13be1c6c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:58.068744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4276e61-3871-4641-8bbd-84ae13be1c6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:58.070414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4276e61-3871-4641-8bbd-84ae13be1c6c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:58.091975 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 421/825] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 30 19:16:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:58.093111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4276e61-3871-4641-8bbd-84ae13be1c6c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:58.105001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfea60-78fa-4727-a576-120939f4aeaa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:58.125301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:58.150635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32baae-7b52-4225-a1bf-edbd08f89b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:58.151702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32baae-7b52-4225-a1bf-edbd08f89b1a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:58.151989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32baae-7b52-4225-a1bf-edbd08f89b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:58.152261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32baae-7b52-4225-a1bf-edbd08f89b1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:58.153148 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32baae-7b52-4225-a1bf-edbd08f89b1a None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=5952db29e6fc4ddbb9aeb0522497312c) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:58.153590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32baae-7b52-4225-a1bf-edbd08f89b1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:58.155281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32baae-7b52-4225-a1bf-edbd08f89b1a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:58.164519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 405/826] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:16:57 2022] GET /identity/v3/role_assignments?user.id=5952db29e6fc4ddbb9aeb0522497312c&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:58.165614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32baae-7b52-4225-a1bf-edbd08f89b1a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:58.235853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:58.269562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4276e61-3871-4641-8bbd-84ae13be1c6c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:58.389047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7b453-6156-43de-8130-8da6eed9a3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:58.390227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7b453-6156-43de-8130-8da6eed9a3d1 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:58.395863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7b453-6156-43de-8130-8da6eed9a3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:58.396571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7b453-6156-43de-8130-8da6eed9a3d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:58.397973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7b453-6156-43de-8130-8da6eed9a3d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:58.400536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7b453-6156-43de-8130-8da6eed9a3d1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:58.401653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32baae-7b52-4225-a1bf-edbd08f89b1a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:58.452722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 422/827] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:16:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:16:58.460490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3bb54-7f9d-48ae-bc09-8803e37268fa None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:58.460490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3bb54-7f9d-48ae-bc09-8803e37268fa None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:58.460490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3bb54-7f9d-48ae-bc09-8803e37268fa None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:58.460490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 423/828] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:16:58.466239 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea7483-414c-42d0-a34d-17744a516cb8 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:58.466539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea7483-414c-42d0-a34d-17744a516cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:58.472397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea7483-414c-42d0-a34d-17744a516cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:58.485042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b0c14-0429-49dc-85f4-256d72e57775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:58.486357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b0c14-0429-49dc-85f4-256d72e57775 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:58.486589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b0c14-0429-49dc-85f4-256d72e57775 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:58.486854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b0c14-0429-49dc-85f4-256d72e57775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:58.494525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-89ea7483-414c-42d0-a34d-17744a516cb8 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:58.495733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-89ea7483-414c-42d0-a34d-17744a516cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:58.506599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b0c14-0429-49dc-85f4-256d72e57775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:58.512396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b0c14-0429-49dc-85f4-256d72e57775 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0f1b0c14-0429-49dc-85f4-256d72e57775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:58.539674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:58.540759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:58.541693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:58.541693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:58.541693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:58.541693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 406/829] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:58.548957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b0c14-0429-49dc-85f4-256d72e57775 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:58.608481 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:58.641404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 424/830] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:16:58.716413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b0c14-0429-49dc-85f4-256d72e57775 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:58.755395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328ae6c-30ce-4835-bfbd-3624f0e55401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:58.756476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328ae6c-30ce-4835-bfbd-3624f0e55401 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:58.756956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328ae6c-30ce-4835-bfbd-3624f0e55401 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:58.757970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328ae6c-30ce-4835-bfbd-3624f0e55401 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:58.758844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328ae6c-30ce-4835-bfbd-3624f0e55401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:58.759344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328ae6c-30ce-4835-bfbd-3624f0e55401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:58.760961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328ae6c-30ce-4835-bfbd-3624f0e55401 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:58.774660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 407/831] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:58.780527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea7483-414c-42d0-a34d-17744a516cb8 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:58.859668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:58.889778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea7483-414c-42d0-a34d-17744a516cb8 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:58.925525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c37c-c334-4e9e-964a-c97a1473735c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:58.926160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c37c-c334-4e9e-964a-c97a1473735c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:58.926434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c37c-c334-4e9e-964a-c97a1473735c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:58.926700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c37c-c334-4e9e-964a-c97a1473735c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:58.934415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5c37c-c334-4e9e-964a-c97a1473735c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:58.936183 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5c37c-c334-4e9e-964a-c97a1473735c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-dbd5c37c-c334-4e9e-964a-c97a1473735c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:16:58.947611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:58.948612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:58.950575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:16:58.950575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:16:58.950575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:16:58.950575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:16:58.950575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:16:58.950575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:16:58.950575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:16:58.950575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:16:58.950575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 425/832] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:16:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:58.951125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328ae6c-30ce-4835-bfbd-3624f0e55401 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:58.957374 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c37c-c334-4e9e-964a-c97a1473735c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:59.047484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:59.062900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328ae6c-30ce-4835-bfbd-3624f0e55401 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:59.065996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c37c-c334-4e9e-964a-c97a1473735c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:59.100181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be3e7b-d07c-466e-83d3-ccacd4a9a2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:59.101686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be3e7b-d07c-466e-83d3-ccacd4a9a2fe None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:59.102209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be3e7b-d07c-466e-83d3-ccacd4a9a2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:59.102590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b654404-d7fc-4d71-b207-d9fd8765598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:59.102885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be3e7b-d07c-466e-83d3-ccacd4a9a2fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:59.103321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b654404-d7fc-4d71-b207-d9fd8765598f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:59.103783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b654404-d7fc-4d71-b207-d9fd8765598f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:59.104215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b654404-d7fc-4d71-b207-d9fd8765598f None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:59.105000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b654404-d7fc-4d71-b207-d9fd8765598f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:59.105396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b654404-d7fc-4d71-b207-d9fd8765598f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:59.107101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b654404-d7fc-4d71-b207-d9fd8765598f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:59.108810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:59.116516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be3e7b-d07c-466e-83d3-ccacd4a9a2fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-82be3e7b-d07c-466e-83d3-ccacd4a9a2fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:16:59.123204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:59.124166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:16:59.125314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:59.126476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 426/833] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:16:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:59.127522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b654404-d7fc-4d71-b207-d9fd8765598f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:59.128078 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 408/834] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:59.133866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be3e7b-d07c-466e-83d3-ccacd4a9a2fe None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:59.233165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b654404-d7fc-4d71-b207-d9fd8765598f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:59.256804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be3e7b-d07c-466e-83d3-ccacd4a9a2fe None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:59.277695 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ce2ef-e60d-4171-b617-fbc3e7c54e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:59.278557 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ce2ef-e60d-4171-b617-fbc3e7c54e12 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:59.278902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ce2ef-e60d-4171-b617-fbc3e7c54e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:59.279241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ce2ef-e60d-4171-b617-fbc3e7c54e12 None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/b638b8149ae149e3aad92ed51be92588/roles/bd4e99b9d39b4ad2abf66671f7a35d61` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:59.298175 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ce2ef-e60d-4171-b617-fbc3e7c54e12 None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=b638b8149ae149e3aad92ed51be92588, role_id=bd4e99b9d39b4ad2abf66671f7a35d61)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:59.299771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ce2ef-e60d-4171-b617-fbc3e7c54e12 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:59.299969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4755dc-3fd0-41d6-9526-24c7928efa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:59.300696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4755dc-3fd0-41d6-9526-24c7928efa76 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:59.301045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4755dc-3fd0-41d6-9526-24c7928efa76 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:59.301389 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4755dc-3fd0-41d6-9526-24c7928efa76 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:59.302227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4755dc-3fd0-41d6-9526-24c7928efa76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:59.302604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4755dc-3fd0-41d6-9526-24c7928efa76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:59.304166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4755dc-3fd0-41d6-9526-24c7928efa76 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:59.314782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 409/835] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:16:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:59.320467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 427/836] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:16:59 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/b638b8149ae149e3aad92ed51be92588/roles/bd4e99b9d39b4ad2abf66671f7a35d61 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:16:59.325892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4755dc-3fd0-41d6-9526-24c7928efa76 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:59.343265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:59.459904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4755dc-3fd0-41d6-9526-24c7928efa76 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:59.502869 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72aa84-0cb0-4263-889f-faa290e5696b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:59.503517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72aa84-0cb0-4263-889f-faa290e5696b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:59.503794 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72aa84-0cb0-4263-889f-faa290e5696b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:59.504063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72aa84-0cb0-4263-889f-faa290e5696b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:59.516649 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72aa84-0cb0-4263-889f-faa290e5696b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:59.517702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72aa84-0cb0-4263-889f-faa290e5696b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-aa72aa84-0cb0-4263-889f-faa290e5696b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:16:59.533315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:59.534299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:16:59.535214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:16:59.535214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:16:59.535214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:59.535214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 410/837] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:59.545337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ce2ef-e60d-4171-b617-fbc3e7c54e12 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:59.596745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:59.653270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ce2ef-e60d-4171-b617-fbc3e7c54e12 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:59.689822 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f022784-e938-46cc-aa13-40e902935ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:59.690454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f022784-e938-46cc-aa13-40e902935ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:59.690730 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f022784-e938-46cc-aa13-40e902935ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:59.691001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f022784-e938-46cc-aa13-40e902935ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:59.691669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f022784-e938-46cc-aa13-40e902935ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:16:59.691971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f022784-e938-46cc-aa13-40e902935ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:59.693520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f022784-e938-46cc-aa13-40e902935ba4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:59.707717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 428/838] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:16:59.713531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72aa84-0cb0-4263-889f-faa290e5696b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:59.783204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:16:59.806036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da520698-1960-4760-87d0-ab38e653353e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:59.806345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da520698-1960-4760-87d0-ab38e653353e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:59.806583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da520698-1960-4760-87d0-ab38e653353e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:59.808118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 429/839] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:16:59.813462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972e504-1ebe-456a-949e-79ede456e672 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:59.813929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972e504-1ebe-456a-949e-79ede456e672 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:59.814207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972e504-1ebe-456a-949e-79ede456e672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:59.830676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72aa84-0cb0-4263-889f-faa290e5696b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:16:59.839518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c972e504-1ebe-456a-949e-79ede456e672 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:16:59.841279 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c972e504-1ebe-456a-949e-79ede456e672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:16:59.866940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d2e91-b40e-4b1b-bb61-9c8fc38a60aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:16:59.867489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2e91-b40e-4b1b-bb61-9c8fc38a60aa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:16:59.867768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2e91-b40e-4b1b-bb61-9c8fc38a60aa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:16:59.868038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2e91-b40e-4b1b-bb61-9c8fc38a60aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:16:59.876572 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d2e91-b40e-4b1b-bb61-9c8fc38a60aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:16:59.878018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d2e91-b40e-4b1b-bb61-9c8fc38a60aa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-865d2e91-b40e-4b1b-bb61-9c8fc38a60aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:16:59.892343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:16:59.893397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:16:59.894450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:16:59.894450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:16:59.894450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:16:59.894450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:16:59.894450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:16:59.894450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:16:59.894450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:16:59.894450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:16:59.894450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 19:16:59.894450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:16:59.894450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 411/840] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:16:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:16:59.900616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d2e91-b40e-4b1b-bb61-9c8fc38a60aa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:16:59.928503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 430/841] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:16:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:00.012785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d2e91-b40e-4b1b-bb61-9c8fc38a60aa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:00.035159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:00.048729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570fd0e-a15f-4e64-af01-2e3b463aab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:00.049331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570fd0e-a15f-4e64-af01-2e3b463aab92 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:00.049609 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570fd0e-a15f-4e64-af01-2e3b463aab92 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:00.049875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570fd0e-a15f-4e64-af01-2e3b463aab92 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:00.050593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570fd0e-a15f-4e64-af01-2e3b463aab92 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:00.050887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570fd0e-a15f-4e64-af01-2e3b463aab92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:00.052304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570fd0e-a15f-4e64-af01-2e3b463aab92 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:00.070764 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 412/842] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Sep 30 19:16:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:00.076458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972e504-1ebe-456a-949e-79ede456e672 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:00.112056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:00.178072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972e504-1ebe-456a-949e-79ede456e672 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:00.212580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7375ce3-4932-4788-80bf-e340623a6e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:00.213179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7375ce3-4932-4788-80bf-e340623a6e1f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:00.213449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7375ce3-4932-4788-80bf-e340623a6e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:00.213768 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7375ce3-4932-4788-80bf-e340623a6e1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:00.236029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570fd0e-a15f-4e64-af01-2e3b463aab92 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:00.289935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7375ce3-4932-4788-80bf-e340623a6e1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:00.291720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7375ce3-4932-4788-80bf-e340623a6e1f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:00.328445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:00.349003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570fd0e-a15f-4e64-af01-2e3b463aab92 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:00.389474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eef104-7d45-493d-8812-1534ca75ce25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:00.390472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eef104-7d45-493d-8812-1534ca75ce25 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:00.390897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eef104-7d45-493d-8812-1534ca75ce25 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:00.391403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eef104-7d45-493d-8812-1534ca75ce25 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:00.392130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eef104-7d45-493d-8812-1534ca75ce25 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:00.393769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eef104-7d45-493d-8812-1534ca75ce25 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d2eef104-7d45-493d-8812-1534ca75ce25 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:00.404163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:00.405484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 19:17:00.406543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:00.408594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 431/843] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:17:00.414170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7375ce3-4932-4788-80bf-e340623a6e1f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:00.414387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 413/844] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:17:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:00.415794 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eef104-7d45-493d-8812-1534ca75ce25 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:00.466318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:00.517284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7375ce3-4932-4788-80bf-e340623a6e1f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:00.523635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eef104-7d45-493d-8812-1534ca75ce25 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:00.555441 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11df224-8a9b-40f7-9cd7-ba05da5492dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:00.556509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11df224-8a9b-40f7-9cd7-ba05da5492dd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:00.557118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11df224-8a9b-40f7-9cd7-ba05da5492dd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:00.557468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11df224-8a9b-40f7-9cd7-ba05da5492dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:00.559543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:00.561944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf56836-e637-4f9d-85aa-9f9a2650fe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:00.562753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf56836-e637-4f9d-85aa-9f9a2650fe3a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:00.563206 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf56836-e637-4f9d-85aa-9f9a2650fe3a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:00.563911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf56836-e637-4f9d-85aa-9f9a2650fe3a None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:00.564762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf56836-e637-4f9d-85aa-9f9a2650fe3a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:00.565234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf56836-e637-4f9d-85aa-9f9a2650fe3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:00.567294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf56836-e637-4f9d-85aa-9f9a2650fe3a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:00.569652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11df224-8a9b-40f7-9cd7-ba05da5492dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:00.572281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11df224-8a9b-40f7-9cd7-ba05da5492dd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:00.573997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 414/845] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:17:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:00.579610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf56836-e637-4f9d-85aa-9f9a2650fe3a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c11df224-8a9b-40f7-9cd7-ba05da5492dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:00.590327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:00.591512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:00.592545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:00.592545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:00.592545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:00.598860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 432/846] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:00.600035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11df224-8a9b-40f7-9cd7-ba05da5492dd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:00.687781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf56836-e637-4f9d-85aa-9f9a2650fe3a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:00.699013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:00.724840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11df224-8a9b-40f7-9cd7-ba05da5492dd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:00.728934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001ab78-ec73-4ee3-b77f-226b3fb516c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:00.729653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001ab78-ec73-4ee3-b77f-226b3fb516c0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:00.730001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001ab78-ec73-4ee3-b77f-226b3fb516c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:00.730347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001ab78-ec73-4ee3-b77f-226b3fb516c0 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:00.731118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001ab78-ec73-4ee3-b77f-226b3fb516c0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:00.731489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001ab78-ec73-4ee3-b77f-226b3fb516c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:00.733024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001ab78-ec73-4ee3-b77f-226b3fb516c0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:00.738928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 415/847] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:17:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:00.766470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9630a21-d881-4b51-923a-7683da2e41e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:00.767421 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630a21-d881-4b51-923a-7683da2e41e1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:00.767898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630a21-d881-4b51-923a-7683da2e41e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:00.768343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630a21-d881-4b51-923a-7683da2e41e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:00.769247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9630a21-d881-4b51-923a-7683da2e41e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:00.769725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9630a21-d881-4b51-923a-7683da2e41e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:00.771296 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9630a21-d881-4b51-923a-7683da2e41e1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:00.785837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 433/848] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:00.794272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001ab78-ec73-4ee3-b77f-226b3fb516c0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:00.824929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c341d7d-3d44-4fa1-aa98-5a7078c45f21 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:00.825507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c341d7d-3d44-4fa1-aa98-5a7078c45f21 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:00.825962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c341d7d-3d44-4fa1-aa98-5a7078c45f21 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:00.827867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 434/849] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:00.833307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29757e1-d582-4bf7-a046-59c23de74f34 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:00.833885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29757e1-d582-4bf7-a046-59c23de74f34 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:00.834341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29757e1-d582-4bf7-a046-59c23de74f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:00.861526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f29757e1-d582-4bf7-a046-59c23de74f34 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:00.862856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f29757e1-d582-4bf7-a046-59c23de74f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:00.914437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:00.924549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001ab78-ec73-4ee3-b77f-226b3fb516c0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:00.947250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 435/850] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:00.963894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22766383-dad9-4ef9-927c-197c60a6ba87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:00.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22766383-dad9-4ef9-927c-197c60a6ba87 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:00.964875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22766383-dad9-4ef9-927c-197c60a6ba87 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:00.965145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22766383-dad9-4ef9-927c-197c60a6ba87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:00.971894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22766383-dad9-4ef9-927c-197c60a6ba87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:00.973518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22766383-dad9-4ef9-927c-197c60a6ba87 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-22766383-dad9-4ef9-927c-197c60a6ba87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:17:00.985274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:00.986185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:00.987129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:17:00.987129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:17:00.987129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:17:00.987129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:17:00.987129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:17:00.987129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:17:00.987129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:00.987129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:00.987129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 416/851] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:00.992556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29757e1-d582-4bf7-a046-59c23de74f34 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:01.094356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29757e1-d582-4bf7-a046-59c23de74f34 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:01.129855 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ee289-64b6-44bb-862b-b1d64df29b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:01.130461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ee289-64b6-44bb-862b-b1d64df29b20 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:01.130732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ee289-64b6-44bb-862b-b1d64df29b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:01.130999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ee289-64b6-44bb-862b-b1d64df29b20 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:01.131729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ee289-64b6-44bb-862b-b1d64df29b20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:01.132033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ee289-64b6-44bb-862b-b1d64df29b20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:01.134060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ee289-64b6-44bb-862b-b1d64df29b20 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:01.146017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 436/852] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:17:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:01.152721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22766383-dad9-4ef9-927c-197c60a6ba87 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:01.185831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:01.220119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ee289-64b6-44bb-862b-b1d64df29b20 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:01.256471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:01.272483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22766383-dad9-4ef9-927c-197c60a6ba87 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:01.315519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc9120-a1d8-44b7-89c8-f318b4459d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:01.316127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc9120-a1d8-44b7-89c8-f318b4459d1e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:01.316461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc9120-a1d8-44b7-89c8-f318b4459d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:01.316737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc9120-a1d8-44b7-89c8-f318b4459d1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:01.317907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc9120-a1d8-44b7-89c8-f318b4459d1e None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=b8785c8c5ac84393afcaf88e891f3b7a) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:01.318218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc9120-a1d8-44b7-89c8-f318b4459d1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:01.320463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc9120-a1d8-44b7-89c8-f318b4459d1e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:01.333398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ee289-64b6-44bb-862b-b1d64df29b20 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:01.333797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 417/853] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:17:01 2022] GET /identity/v3/role_assignments?user.id=b8785c8c5ac84393afcaf88e891f3b7a&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:01.375074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149b9df-a655-4e81-9d0f-a9621bc7289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:01.375730 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149b9df-a655-4e81-9d0f-a9621bc7289f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:01.376004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149b9df-a655-4e81-9d0f-a9621bc7289f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:01.376274 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149b9df-a655-4e81-9d0f-a9621bc7289f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:01.376899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149b9df-a655-4e81-9d0f-a9621bc7289f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:01.378276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149b9df-a655-4e81-9d0f-a9621bc7289f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d149b9df-a655-4e81-9d0f-a9621bc7289f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:01.389223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:01.390244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:01.391196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 437/854] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Sep 30 19:17:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:01.395251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc9120-a1d8-44b7-89c8-f318b4459d1e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:01.508868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:01.515308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc9120-a1d8-44b7-89c8-f318b4459d1e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:01.555621 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2fc5c-1cd7-4b12-9811-d68b86b2ce9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:01.556260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fc5c-1cd7-4b12-9811-d68b86b2ce9e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:01.556554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fc5c-1cd7-4b12-9811-d68b86b2ce9e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:01.556821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fc5c-1cd7-4b12-9811-d68b86b2ce9e None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:01.557476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2fc5c-1cd7-4b12-9811-d68b86b2ce9e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:01.557774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2fc5c-1cd7-4b12-9811-d68b86b2ce9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:01.559190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2fc5c-1cd7-4b12-9811-d68b86b2ce9e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:01.565265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 418/855] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 30 19:17:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:01.571673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149b9df-a655-4e81-9d0f-a9621bc7289f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:01.670814 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd12bf-1a71-4974-896c-67264da04ef5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:01.671096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd12bf-1a71-4974-896c-67264da04ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:01.671366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd12bf-1a71-4974-896c-67264da04ef5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:01.673159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 419/856] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:01.678770 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9eccc-486b-4341-a58b-55932c36017e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:01.679813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9eccc-486b-4341-a58b-55932c36017e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:01.680233 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9eccc-486b-4341-a58b-55932c36017e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:01.685045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:01.688826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149b9df-a655-4e81-9d0f-a9621bc7289f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:01.701839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-22a9eccc-486b-4341-a58b-55932c36017e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:01.703282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-22a9eccc-486b-4341-a58b-55932c36017e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:01.738251 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf40dc-7a03-4302-9483-3626819457df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:01.739018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf40dc-7a03-4302-9483-3626819457df None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:01.739366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf40dc-7a03-4302-9483-3626819457df None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:01.739705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf40dc-7a03-4302-9483-3626819457df None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:01.740444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf40dc-7a03-4302-9483-3626819457df None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:01.740812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf40dc-7a03-4302-9483-3626819457df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:01.742292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf40dc-7a03-4302-9483-3626819457df None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:01.748017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 438/857] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 30 19:17:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:01.804885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 420/858] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:02.078255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf40dc-7a03-4302-9483-3626819457df None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:02.183637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf40dc-7a03-4302-9483-3626819457df None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:02.218193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32131ec-cb2a-4377-8d84-def380fbf112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:02.218823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32131ec-cb2a-4377-8d84-def380fbf112 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:02.219097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32131ec-cb2a-4377-8d84-def380fbf112 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:02.219369 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32131ec-cb2a-4377-8d84-def380fbf112 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:02.230402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32131ec-cb2a-4377-8d84-def380fbf112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a32131ec-cb2a-4377-8d84-def380fbf112 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:17:02.235675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:02.236679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:02.237933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 439/859] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:17:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:02.243464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9eccc-486b-4341-a58b-55932c36017e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:02.335918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:02.346079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9eccc-486b-4341-a58b-55932c36017e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:02.382867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9719c-db94-4f5e-80a8-64a8dfe60fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:02.383473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9719c-db94-4f5e-80a8-64a8dfe60fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:02.383748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9719c-db94-4f5e-80a8-64a8dfe60fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:02.384017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9719c-db94-4f5e-80a8-64a8dfe60fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:02.384784 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9719c-db94-4f5e-80a8-64a8dfe60fb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:02.385086 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9719c-db94-4f5e-80a8-64a8dfe60fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:02.386453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9719c-db94-4f5e-80a8-64a8dfe60fb4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:02.395878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 421/860] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:17:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:02.402306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32131ec-cb2a-4377-8d84-def380fbf112 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:02.431770 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:02.511762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32131ec-cb2a-4377-8d84-def380fbf112 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:02.546352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ceac87-9aba-477b-8117-50cd36a4afce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:02.546981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ceac87-9aba-477b-8117-50cd36a4afce None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:02.547255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ceac87-9aba-477b-8117-50cd36a4afce None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:02.547525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ceac87-9aba-477b-8117-50cd36a4afce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:02.559031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ceac87-9aba-477b-8117-50cd36a4afce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:02.560485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ceac87-9aba-477b-8117-50cd36a4afce None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-39ceac87-9aba-477b-8117-50cd36a4afce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:02.575461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:02.576468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:02.577843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:02.577843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:02.577843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:02.577843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 440/861] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:02.582699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9719c-db94-4f5e-80a8-64a8dfe60fb4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:02.685019 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9719c-db94-4f5e-80a8-64a8dfe60fb4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:02.724687 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffde229-70d2-47cf-b7ce-4e283ad569bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:02.725659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffde229-70d2-47cf-b7ce-4e283ad569bc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:02.726704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffde229-70d2-47cf-b7ce-4e283ad569bc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:02.727273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffde229-70d2-47cf-b7ce-4e283ad569bc None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:02.728111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffde229-70d2-47cf-b7ce-4e283ad569bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:02.728666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffde229-70d2-47cf-b7ce-4e283ad569bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:02.730223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffde229-70d2-47cf-b7ce-4e283ad569bc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:02.743062 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 422/862] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:02.750790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ceac87-9aba-477b-8117-50cd36a4afce None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:02.827655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:02.850826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ceac87-9aba-477b-8117-50cd36a4afce None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:02.887965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bbf6c-2411-4677-85a5-cfe81ae97680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:02.888645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bbf6c-2411-4677-85a5-cfe81ae97680 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:02.888922 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bbf6c-2411-4677-85a5-cfe81ae97680 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:02.889189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bbf6c-2411-4677-85a5-cfe81ae97680 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:02.897997 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bbf6c-2411-4677-85a5-cfe81ae97680 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:02.899455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bbf6c-2411-4677-85a5-cfe81ae97680 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a65bbf6c-2411-4677-85a5-cfe81ae97680 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:17:02.911532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:02.912844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:02.914981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:02.914981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:02.914981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:17:02.914981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:17:02.914981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:17:02.914981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:17:02.914981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:17:02.914981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:17:02.914981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:17:02.914981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:02.914981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 441/863] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 30 19:17:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:02.919506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffde229-70d2-47cf-b7ce-4e283ad569bc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:02.935292 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:02.992733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3d8e6-7ef2-4b66-b270-881f37692ce9 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:02.993070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3d8e6-7ef2-4b66-b270-881f37692ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:02.993276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3d8e6-7ef2-4b66-b270-881f37692ce9 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:02.995954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 442/864] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:03.004425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965a192-8119-4a49-b41c-212866dbf5b6 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:03.004815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965a192-8119-4a49-b41c-212866dbf5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:03.005187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965a192-8119-4a49-b41c-212866dbf5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:03.024982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e965a192-8119-4a49-b41c-212866dbf5b6 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:03.026300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffde229-70d2-47cf-b7ce-4e283ad569bc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:03.026423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e965a192-8119-4a49-b41c-212866dbf5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:03.070921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b4ee1-fab4-40ce-bc5b-4fa5a9efa14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:03.071533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b4ee1-fab4-40ce-bc5b-4fa5a9efa14f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:03.071969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b4ee1-fab4-40ce-bc5b-4fa5a9efa14f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:03.072246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b4ee1-fab4-40ce-bc5b-4fa5a9efa14f None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:03.073932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b4ee1-fab4-40ce-bc5b-4fa5a9efa14f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:03.074249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b4ee1-fab4-40ce-bc5b-4fa5a9efa14f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:03.075766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b4ee1-fab4-40ce-bc5b-4fa5a9efa14f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:03.099414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 423/865] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 30 19:17:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:03.105480 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b4ee1-fab4-40ce-bc5b-4fa5a9efa14f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:03.108925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 443/866] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:03.179689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:03.212623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b4ee1-fab4-40ce-bc5b-4fa5a9efa14f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:03.247442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4626870-757a-481e-9b49-2ce86c0a559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:03.248045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4626870-757a-481e-9b49-2ce86c0a559c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:03.248324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4626870-757a-481e-9b49-2ce86c0a559c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:03.248624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4626870-757a-481e-9b49-2ce86c0a559c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:03.318416 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4626870-757a-481e-9b49-2ce86c0a559c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:03.320431 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4626870-757a-481e-9b49-2ce86c0a559c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:03.382444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965a192-8119-4a49-b41c-212866dbf5b6 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:03.392401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:03.441316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 424/867] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:03 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:17:03.449409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4626870-757a-481e-9b49-2ce86c0a559c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:03.465063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:03.511859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965a192-8119-4a49-b41c-212866dbf5b6 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:03.558205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d9730-dd56-48fb-83de-7ef866987dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:03.558822 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d9730-dd56-48fb-83de-7ef866987dfd None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:03.559105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d9730-dd56-48fb-83de-7ef866987dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:03.559420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d9730-dd56-48fb-83de-7ef866987dfd None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:03.559938 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4626870-757a-481e-9b49-2ce86c0a559c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:03.560101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d9730-dd56-48fb-83de-7ef866987dfd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:03.561749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d9730-dd56-48fb-83de-7ef866987dfd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:03.589318 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 444/868] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:17:03 2022] POST /identity/v3/services => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:17:03.612693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef38f2b-762a-43a8-be44-b097bcc2905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:03.613358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef38f2b-762a-43a8-be44-b097bcc2905f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:03.613638 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef38f2b-762a-43a8-be44-b097bcc2905f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:03.613973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef38f2b-762a-43a8-be44-b097bcc2905f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:03.627995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef38f2b-762a-43a8-be44-b097bcc2905f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:03.629755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef38f2b-762a-43a8-be44-b097bcc2905f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2ef38f2b-762a-43a8-be44-b097bcc2905f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:03.647769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:03.649742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:03.650896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:03.650896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:03.650896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:03.650896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 425/869] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:03.655890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d9730-dd56-48fb-83de-7ef866987dfd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:03.736176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ada16-963e-4a17-85e8-747c9710e3f5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:03.736622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ada16-963e-4a17-85e8-747c9710e3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:03.736622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ada16-963e-4a17-85e8-747c9710e3f5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:03.738551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 426/870] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:03.744840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844e7fd-c5b4-4d70-98d2-9ddc23eea296 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:03.745048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844e7fd-c5b4-4d70-98d2-9ddc23eea296 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:03.745299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844e7fd-c5b4-4d70-98d2-9ddc23eea296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:03.765457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f844e7fd-c5b4-4d70-98d2-9ddc23eea296 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:03.766738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f844e7fd-c5b4-4d70-98d2-9ddc23eea296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:03.781771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:03.785520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d9730-dd56-48fb-83de-7ef866987dfd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:03.822526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:03.830798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f32625-1d86-4893-b6de-8f31f5d155f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:03.831417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f32625-1d86-4893-b6de-8f31f5d155f0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:03.831701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f32625-1d86-4893-b6de-8f31f5d155f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:03.831980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f32625-1d86-4893-b6de-8f31f5d155f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:03.832661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f32625-1d86-4893-b6de-8f31f5d155f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:03.832965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f32625-1d86-4893-b6de-8f31f5d155f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:03.834820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f32625-1d86-4893-b6de-8f31f5d155f0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:03.848430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 445/871] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:03.854979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f32625-1d86-4893-b6de-8f31f5d155f0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:03.866340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 427/872] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:03.975238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f32625-1d86-4893-b6de-8f31f5d155f0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:03.997612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5ba2c-6568-43a5-9cad-ad8e0a3fc55c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:03.997969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5ba2c-6568-43a5-9cad-ad8e0a3fc55c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:03.998359 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5ba2c-6568-43a5-9cad-ad8e0a3fc55c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:04.000080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 428/873] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:04.004880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8469384-6a91-4d92-864f-a20dac2a4eae None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:04.005151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8469384-6a91-4d92-864f-a20dac2a4eae None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:04.005421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8469384-6a91-4d92-864f-a20dac2a4eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:04.014764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b5aef-67ca-418e-99d8-32a39fbbadcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:04.015404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b5aef-67ca-418e-99d8-32a39fbbadcd None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:04.015725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b5aef-67ca-418e-99d8-32a39fbbadcd None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:04.016063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b5aef-67ca-418e-99d8-32a39fbbadcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:04.023835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b5aef-67ca-418e-99d8-32a39fbbadcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:04.024459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e8469384-6a91-4d92-864f-a20dac2a4eae None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:04.025304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b5aef-67ca-418e-99d8-32a39fbbadcd None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:04.025622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e8469384-6a91-4d92-864f-a20dac2a4eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-cf5b5aef-67ca-418e-99d8-32a39fbbadcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:17:04.040334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:04.042024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:04.043417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:17:04.043417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:17:04.043417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:17:04.043417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:17:04.043417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:17:04.043417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:17:04.043417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:04.043417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:04.043417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 446/874] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:04.047171 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b5aef-67ca-418e-99d8-32a39fbbadcd None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:04.099946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:04.108840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 429/875] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:04.154792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8469384-6a91-4d92-864f-a20dac2a4eae None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:04.155313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b5aef-67ca-418e-99d8-32a39fbbadcd None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:04.191999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a76fb-6fa9-4066-a546-254a87663164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:04.192639 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a76fb-6fa9-4066-a546-254a87663164 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:04.192925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a76fb-6fa9-4066-a546-254a87663164 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:04.193199 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a76fb-6fa9-4066-a546-254a87663164 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:04.193893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a76fb-6fa9-4066-a546-254a87663164 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:04.194196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a76fb-6fa9-4066-a546-254a87663164 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:04.196329 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a76fb-6fa9-4066-a546-254a87663164 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:04.209688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 447/876] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:17:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:04.215591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a76fb-6fa9-4066-a546-254a87663164 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:04.245219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:04.273349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8469384-6a91-4d92-864f-a20dac2a4eae None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:04.323415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a76fb-6fa9-4066-a546-254a87663164 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:04.326821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a5f37-f5b7-4e6f-b094-825bc78d5ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:04.327721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a5f37-f5b7-4e6f-b094-825bc78d5ecf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:04.328217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a5f37-f5b7-4e6f-b094-825bc78d5ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:04.328725 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a5f37-f5b7-4e6f-b094-825bc78d5ecf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:04.329523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a5f37-f5b7-4e6f-b094-825bc78d5ecf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:04.331564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a5f37-f5b7-4e6f-b094-825bc78d5ecf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-912a5f37-f5b7-4e6f-b094-825bc78d5ecf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:04.343809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:04.345000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:04.345915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:04.346741 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:04.354137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 430/877] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 30 19:17:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:04.355851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a5f37-f5b7-4e6f-b094-825bc78d5ecf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:04.368238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a53beb-f21b-4580-826c-2a6ab8727bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:04.368651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a53beb-f21b-4580-826c-2a6ab8727bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:04.369002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a53beb-f21b-4580-826c-2a6ab8727bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:04.369343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a53beb-f21b-4580-826c-2a6ab8727bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:04.370140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a53beb-f21b-4580-826c-2a6ab8727bc3 None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=b638b8149ae149e3aad92ed51be92588) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:04.370513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a53beb-f21b-4580-826c-2a6ab8727bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:04.372073 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a53beb-f21b-4580-826c-2a6ab8727bc3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:04.384758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 448/878] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:17:04 2022] GET /identity/v3/role_assignments?user.id=b638b8149ae149e3aad92ed51be92588&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:04.395412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a53beb-f21b-4580-826c-2a6ab8727bc3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:04.498295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a5f37-f5b7-4e6f-b094-825bc78d5ecf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:04.512971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a53beb-f21b-4580-826c-2a6ab8727bc3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:04.547889 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0518f59-2c46-452b-8016-aea9e41e0509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:04.552398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0518f59-2c46-452b-8016-aea9e41e0509 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:04.552398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0518f59-2c46-452b-8016-aea9e41e0509 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:04.552398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0518f59-2c46-452b-8016-aea9e41e0509 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:04.552398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0518f59-2c46-452b-8016-aea9e41e0509 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:04.552398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0518f59-2c46-452b-8016-aea9e41e0509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:04.552398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0518f59-2c46-452b-8016-aea9e41e0509 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:04.559043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 431/879] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 30 19:17:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:04.559497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf4074-4ffa-41d6-b77a-dc9d9a50358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:04.560293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf4074-4ffa-41d6-b77a-dc9d9a50358a None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:04.560430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf4074-4ffa-41d6-b77a-dc9d9a50358a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:04.560661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf4074-4ffa-41d6-b77a-dc9d9a50358a None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:04.561298 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf4074-4ffa-41d6-b77a-dc9d9a50358a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:04.562689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf4074-4ffa-41d6-b77a-dc9d9a50358a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:04.565277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0518f59-2c46-452b-8016-aea9e41e0509 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:04.578688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:04.583549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 449/880] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:17:04 2022] POST /identity/v3/services => generated 269 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:17:04.695760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0518f59-2c46-452b-8016-aea9e41e0509 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:04.739815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78d645-0958-4b94-bf6d-64b9182835d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:04.740520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78d645-0958-4b94-bf6d-64b9182835d6 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:04.740733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78d645-0958-4b94-bf6d-64b9182835d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:04.741000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78d645-0958-4b94-bf6d-64b9182835d6 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:04.742176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78d645-0958-4b94-bf6d-64b9182835d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:04.742474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78d645-0958-4b94-bf6d-64b9182835d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:04.743875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78d645-0958-4b94-bf6d-64b9182835d6 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:04.749439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 432/881] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 30 19:17:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:05.846291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a735075-d76f-4de8-989a-29f455229171 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:05.846856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a735075-d76f-4de8-989a-29f455229171 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:05.846856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a735075-d76f-4de8-989a-29f455229171 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:05.848429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 450/882] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:05.853870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554866b1-10e7-4dc2-8a76-a35ebb51c81a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:05.854270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554866b1-10e7-4dc2-8a76-a35ebb51c81a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:05.854448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554866b1-10e7-4dc2-8a76-a35ebb51c81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:05.872691 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-554866b1-10e7-4dc2-8a76-a35ebb51c81a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:05.873867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-554866b1-10e7-4dc2-8a76-a35ebb51c81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:05.934261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:05.948471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 433/883] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:06.215897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a735075-d76f-4de8-989a-29f455229171 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:06.247601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:06.323798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a735075-d76f-4de8-989a-29f455229171 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:06.358868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679a967-3708-420e-ada2-d99ded69e4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:06.359517 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679a967-3708-420e-ada2-d99ded69e4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:06.359785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679a967-3708-420e-ada2-d99ded69e4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:06.360087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679a967-3708-420e-ada2-d99ded69e4e3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:06.360672 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679a967-3708-420e-ada2-d99ded69e4e3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:06.362066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679a967-3708-420e-ada2-d99ded69e4e3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-2679a967-3708-420e-ada2-d99ded69e4e3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:06.372623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:06.373746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 19:17:06.374652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:06.378764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 451/884] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:17:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:06.380144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554866b1-10e7-4dc2-8a76-a35ebb51c81a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:06.493713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554866b1-10e7-4dc2-8a76-a35ebb51c81a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:06.528546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411b0e8-6760-4b18-af78-e2290cba85b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:06.529425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b0e8-6760-4b18-af78-e2290cba85b2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:06.529861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b0e8-6760-4b18-af78-e2290cba85b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:06.530289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b0e8-6760-4b18-af78-e2290cba85b2 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:06.531075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411b0e8-6760-4b18-af78-e2290cba85b2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:06.531531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411b0e8-6760-4b18-af78-e2290cba85b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:06.533116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411b0e8-6760-4b18-af78-e2290cba85b2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:06.539165 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 434/885] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:17:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:06.544750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679a967-3708-420e-ada2-d99ded69e4e3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:06.649287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:06.658264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679a967-3708-420e-ada2-d99ded69e4e3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:06.701202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667231a-d8c2-4541-a904-86de2365395c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:06.702165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667231a-d8c2-4541-a904-86de2365395c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:06.702647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667231a-d8c2-4541-a904-86de2365395c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:06.703121 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667231a-d8c2-4541-a904-86de2365395c None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:06.704050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667231a-d8c2-4541-a904-86de2365395c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:06.704600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667231a-d8c2-4541-a904-86de2365395c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:06.706213 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667231a-d8c2-4541-a904-86de2365395c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:06.712255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 452/886] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:17:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:06.720573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411b0e8-6760-4b18-af78-e2290cba85b2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:06.766203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1afe2f-88a3-4880-8ce1-f82095430fee None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:06.766833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1afe2f-88a3-4880-8ce1-f82095430fee None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:06.767582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1afe2f-88a3-4880-8ce1-f82095430fee None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:06.770212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 453/887] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:06.775126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6492f-c97d-41f1-a05c-7675b4f99e8d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:06.775435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6492f-c97d-41f1-a05c-7675b4f99e8d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:06.775668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6492f-c97d-41f1-a05c-7675b4f99e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:06.785223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:06.794330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-51f6492f-c97d-41f1-a05c-7675b4f99e8d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:06.795366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-51f6492f-c97d-41f1-a05c-7675b4f99e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:06.831966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411b0e8-6760-4b18-af78-e2290cba85b2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:06.863616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 454/888] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:06.865453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a550-4d54-426a-899d-a329697baffc None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:06.865879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a550-4d54-426a-899d-a329697baffc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:06.866310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a550-4d54-426a-899d-a329697baffc None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:06.872067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5dd36e-e451-49e9-a6f1-1fab8f26cbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:06.872067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5dd36e-e451-49e9-a6f1-1fab8f26cbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:06.872067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5dd36e-e451-49e9-a6f1-1fab8f26cbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:06.872067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5dd36e-e451-49e9-a6f1-1fab8f26cbb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:06.872771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5dd36e-e451-49e9-a6f1-1fab8f26cbb0 None admin] RBAC: Adding query filter params (interface=public, service_id=3bcf2c99750a41868b79048595e9cbd4, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:06.872771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5dd36e-e451-49e9-a6f1-1fab8f26cbb0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:06.872771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5dd36e-e451-49e9-a6f1-1fab8f26cbb0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:06.873285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 455/889] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:06.874853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8076c-1f3b-486b-8ad1-cf5fc01f9c29 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:06.875283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8076c-1f3b-486b-8ad1-cf5fc01f9c29 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:06.876169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8076c-1f3b-486b-8ad1-cf5fc01f9c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:06.880588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 435/890] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 30 19:17:06 2022] GET /identity/v3/endpoints?service_id=3bcf2c99750a41868b79048595e9cbd4&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:06.885364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5dd36e-e451-49e9-a6f1-1fab8f26cbb0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:06.899454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-93f8076c-1f3b-486b-8ad1-cf5fc01f9c29 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:06.900747 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-93f8076c-1f3b-486b-8ad1-cf5fc01f9c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:06.984412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 456/891] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:07.004244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5dd36e-e451-49e9-a6f1-1fab8f26cbb0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:07.029795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:07.043189 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b903a3-9cea-43c5-bd10-65a1a0a01753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:07.044127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b903a3-9cea-43c5-bd10-65a1a0a01753 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:07.044678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b903a3-9cea-43c5-bd10-65a1a0a01753 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:07.045187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b903a3-9cea-43c5-bd10-65a1a0a01753 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:07.045988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b903a3-9cea-43c5-bd10-65a1a0a01753 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:07.047553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b903a3-9cea-43c5-bd10-65a1a0a01753 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:07.057322 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 436/892] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 30 19:17:06 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:17:07.059236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a567e8-0ace-4fd4-ae34-45a46670f5b7 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:07.060014 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a567e8-0ace-4fd4-ae34-45a46670f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:07.060551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a567e8-0ace-4fd4-ae34-45a46670f5b7 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:07.063053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 457/893] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:07.069213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779eea09-8f87-49c2-8f49-c5b2961ced31 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:07.069500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779eea09-8f87-49c2-8f49-c5b2961ced31 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:07.069884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779eea09-8f87-49c2-8f49-c5b2961ced31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:07.092437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-779eea09-8f87-49c2-8f49-c5b2961ced31 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:07.094075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-779eea09-8f87-49c2-8f49-c5b2961ced31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:07.155157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a567e8-0ace-4fd4-ae34-45a46670f5b7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:07.180871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:07.181238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 437/894] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:07.263967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779eea09-8f87-49c2-8f49-c5b2961ced31 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:07.264807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a567e8-0ace-4fd4-ae34-45a46670f5b7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:07.306326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779b959-96e2-45c0-84e2-b226db793fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:07.306889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779b959-96e2-45c0-84e2-b226db793fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:07.307168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779b959-96e2-45c0-84e2-b226db793fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:07.307442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779b959-96e2-45c0-84e2-b226db793fc7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:07.321288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779b959-96e2-45c0-84e2-b226db793fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f779b959-96e2-45c0-84e2-b226db793fc7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:17:07.327414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:07.328599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:07.329655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 458/895] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 30 19:17:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:07.337207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779b959-96e2-45c0-84e2-b226db793fc7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:07.407472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779eea09-8f87-49c2-8f49-c5b2961ced31 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:07.426230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:07.439871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:07.447454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d5808-d4bd-42cf-b8f4-f395278a85a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:07.448541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d5808-d4bd-42cf-b8f4-f395278a85a5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:07.449109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d5808-d4bd-42cf-b8f4-f395278a85a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:07.449569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d5808-d4bd-42cf-b8f4-f395278a85a5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:07.450320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d5808-d4bd-42cf-b8f4-f395278a85a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:07.451939 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d5808-d4bd-42cf-b8f4-f395278a85a5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d03d5808-d4bd-42cf-b8f4-f395278a85a5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:07.461968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:07.463113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 19:17:07.464079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:07.464871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 438/896] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Sep 30 19:17:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:07.468388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d5808-d4bd-42cf-b8f4-f395278a85a5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:07.471601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779b959-96e2-45c0-84e2-b226db793fc7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:07.509685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4f568-6ab9-4dc4-b559-36d9f19bdf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:07.510293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4f568-6ab9-4dc4-b559-36d9f19bdf67 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:07.510567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4f568-6ab9-4dc4-b559-36d9f19bdf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:07.510887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4f568-6ab9-4dc4-b559-36d9f19bdf67 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:07.511485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e4f568-6ab9-4dc4-b559-36d9f19bdf67 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:07.511790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e4f568-6ab9-4dc4-b559-36d9f19bdf67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:07.513169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e4f568-6ab9-4dc4-b559-36d9f19bdf67 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:07.522243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 459/897] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 30 19:17:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:07.528670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4f568-6ab9-4dc4-b559-36d9f19bdf67 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:07.572436 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d5808-d4bd-42cf-b8f4-f395278a85a5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:07.607280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbfbcb-19db-4194-b459-28de47df052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:07.607881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbfbcb-19db-4194-b459-28de47df052a None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:07.608198 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbfbcb-19db-4194-b459-28de47df052a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:07.608497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbfbcb-19db-4194-b459-28de47df052a None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:07.609141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbfbcb-19db-4194-b459-28de47df052a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:07.610501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbfbcb-19db-4194-b459-28de47df052a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:07.625830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 439/898] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:17:07 2022] POST /identity/v3/services => generated 241 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:17:07.637458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbfbcb-19db-4194-b459-28de47df052a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:07.639120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4f568-6ab9-4dc4-b559-36d9f19bdf67 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:07.699322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cb129-6c03-4671-8360-a9fcdaf427bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:07.700022 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:07.701355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cb129-6c03-4671-8360-a9fcdaf427bc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:07.704792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cb129-6c03-4671-8360-a9fcdaf427bc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:07.704792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cb129-6c03-4671-8360-a9fcdaf427bc None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:07.704792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1cb129-6c03-4671-8360-a9fcdaf427bc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:07.704792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1cb129-6c03-4671-8360-a9fcdaf427bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:07.704792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1cb129-6c03-4671-8360-a9fcdaf427bc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:07.708444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:07.715563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 460/899] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 30 19:17:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:07.720323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cb129-6c03-4671-8360-a9fcdaf427bc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:07.780144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbfbcb-19db-4194-b459-28de47df052a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:07.821570 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579c99a-4604-41bf-80cd-8b009c2215f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:07.822864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579c99a-4604-41bf-80cd-8b009c2215f7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:07.823206 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579c99a-4604-41bf-80cd-8b009c2215f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:07.823570 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579c99a-4604-41bf-80cd-8b009c2215f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:07.829859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cb129-6c03-4671-8360-a9fcdaf427bc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:07.836824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579c99a-4604-41bf-80cd-8b009c2215f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:07.838431 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579c99a-4604-41bf-80cd-8b009c2215f7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5579c99a-4604-41bf-80cd-8b009c2215f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:07.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:07.854532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:07.855565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:07.855565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:07.855565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:07.855565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 440/900] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:07.860153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579c99a-4604-41bf-80cd-8b009c2215f7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:07.875808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd82644-4575-48ca-99f2-efa82c91267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:07.876465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd82644-4575-48ca-99f2-efa82c91267d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:07.876727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd82644-4575-48ca-99f2-efa82c91267d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:07.876998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd82644-4575-48ca-99f2-efa82c91267d None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:07.877631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd82644-4575-48ca-99f2-efa82c91267d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:07.877928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd82644-4575-48ca-99f2-efa82c91267d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:07.879505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd82644-4575-48ca-99f2-efa82c91267d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:07.886191 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 461/901] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 30 19:17:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:07.892238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd82644-4575-48ca-99f2-efa82c91267d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:07.962450 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:07.970633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:07.985375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579c99a-4604-41bf-80cd-8b009c2215f7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:08.013114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd82644-4575-48ca-99f2-efa82c91267d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:08.021620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b7a69-04e1-4d59-b89f-ea14e0db04d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:08.022227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b7a69-04e1-4d59-b89f-ea14e0db04d4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:08.022568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b7a69-04e1-4d59-b89f-ea14e0db04d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:08.022876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b7a69-04e1-4d59-b89f-ea14e0db04d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:08.023515 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b7a69-04e1-4d59-b89f-ea14e0db04d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:08.023813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b7a69-04e1-4d59-b89f-ea14e0db04d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:08.025220 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b7a69-04e1-4d59-b89f-ea14e0db04d4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:08.042348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 441/902] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:08.043397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b7a69-04e1-4d59-b89f-ea14e0db04d4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:08.062751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047c960-96ce-480b-a5db-8513cc75b581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:08.063768 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047c960-96ce-480b-a5db-8513cc75b581 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:08.064239 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047c960-96ce-480b-a5db-8513cc75b581 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:08.065001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047c960-96ce-480b-a5db-8513cc75b581 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:08.066294 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047c960-96ce-480b-a5db-8513cc75b581 None admin] RBAC: Adding query filter params (interface=public, service_id=cee36acf4dd547ba9888e8676b25c224, region_id=RegionOne) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:08.066821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047c960-96ce-480b-a5db-8513cc75b581 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:08.068461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047c960-96ce-480b-a5db-8513cc75b581 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:08.080046 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 462/903] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 30 19:17:07 2022] GET /identity/v3/endpoints?service_id=cee36acf4dd547ba9888e8676b25c224&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:08.080832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047c960-96ce-480b-a5db-8513cc75b581 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:08.173024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b7a69-04e1-4d59-b89f-ea14e0db04d4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:08.206691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047c960-96ce-480b-a5db-8513cc75b581 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:08.208512 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35798a12-29dc-471f-a6ec-9539d840badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:08.209251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798a12-29dc-471f-a6ec-9539d840badc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:08.209630 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798a12-29dc-471f-a6ec-9539d840badc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:08.210780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798a12-29dc-471f-a6ec-9539d840badc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:08.220477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35798a12-29dc-471f-a6ec-9539d840badc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:08.222444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35798a12-29dc-471f-a6ec-9539d840badc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:08.237060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-35798a12-29dc-471f-a6ec-9539d840badc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:17:08.242256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:08.243398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:08.244602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:08.244602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:08.244602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:17:08.244602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:17:08.244602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:17:08.244602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:17:08.244602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:17:08.244602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:17:08.244602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:17:08.244602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:08.252067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 442/904] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 30 19:17:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:08.252911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35798a12-29dc-471f-a6ec-9539d840badc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:08.254238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ddf3d-fdc2-420f-875c-f4faacb6adea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:08.255096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ddf3d-fdc2-420f-875c-f4faacb6adea None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:08.255875 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ddf3d-fdc2-420f-875c-f4faacb6adea None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:08.256361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ddf3d-fdc2-420f-875c-f4faacb6adea None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:08.258235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ddf3d-fdc2-420f-875c-f4faacb6adea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:08.259818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ddf3d-fdc2-420f-875c-f4faacb6adea None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:08.273633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 463/905] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 30 19:17:08 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:17:08.274939 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:08.371339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35798a12-29dc-471f-a6ec-9539d840badc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:08.411304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbed92-cb79-47b0-a340-9439b81607e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:08.411925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbed92-cb79-47b0-a340-9439b81607e9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:08.412203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbed92-cb79-47b0-a340-9439b81607e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:08.412745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbed92-cb79-47b0-a340-9439b81607e9 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:08.413576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbed92-cb79-47b0-a340-9439b81607e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:08.413875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbed92-cb79-47b0-a340-9439b81607e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:08.415316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbed92-cb79-47b0-a340-9439b81607e9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:08.436002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 443/906] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 30 19:17:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:08.441844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ddf3d-fdc2-420f-875c-f4faacb6adea None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:08.549700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ddf3d-fdc2-420f-875c-f4faacb6adea None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:08.584864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a933c-ad05-4cec-addd-489cb128628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:08.585558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a933c-ad05-4cec-addd-489cb128628c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:08.585842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a933c-ad05-4cec-addd-489cb128628c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:08.586115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a933c-ad05-4cec-addd-489cb128628c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:08.648222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:08.658523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a933c-ad05-4cec-addd-489cb128628c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:08.659995 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a933c-ad05-4cec-addd-489cb128628c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:08.761262 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 464/907] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:17:08.766552 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbed92-cb79-47b0-a340-9439b81607e9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:08.838206 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:08.870342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbed92-cb79-47b0-a340-9439b81607e9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:08.905254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3b09d-ec52-4dc2-b6a6-18777a595566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:08.905802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3b09d-ec52-4dc2-b6a6-18777a595566 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:08.906075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3b09d-ec52-4dc2-b6a6-18777a595566 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:08.906340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3b09d-ec52-4dc2-b6a6-18777a595566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:08.918103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3b09d-ec52-4dc2-b6a6-18777a595566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:08.919531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3b09d-ec52-4dc2-b6a6-18777a595566 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f4c3b09d-ec52-4dc2-b6a6-18777a595566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:08.934266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:08.935254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:08.936207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:08.936207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:08.936207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:08.936207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 444/908] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:08.941188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a933c-ad05-4cec-addd-489cb128628c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:09.044273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a933c-ad05-4cec-addd-489cb128628c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:09.053865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:09.079709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872104e-43a7-4be5-a0dd-ddb6ba73ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:09.080314 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872104e-43a7-4be5-a0dd-ddb6ba73ce5d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:09.080618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872104e-43a7-4be5-a0dd-ddb6ba73ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:09.080883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872104e-43a7-4be5-a0dd-ddb6ba73ce5d None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:09.081516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872104e-43a7-4be5-a0dd-ddb6ba73ce5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:09.081812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872104e-43a7-4be5-a0dd-ddb6ba73ce5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:09.083219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872104e-43a7-4be5-a0dd-ddb6ba73ce5d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:09.095513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 465/909] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:09.100810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3b09d-ec52-4dc2-b6a6-18777a595566 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:09.206841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3b09d-ec52-4dc2-b6a6-18777a595566 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:09.232584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:09.243229 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f762b8-43ed-4289-a4d2-df6936424cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:09.243853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f762b8-43ed-4289-a4d2-df6936424cbc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:09.244125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f762b8-43ed-4289-a4d2-df6936424cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:09.244412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f762b8-43ed-4289-a4d2-df6936424cbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:09.251461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f762b8-43ed-4289-a4d2-df6936424cbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:09.252931 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f762b8-43ed-4289-a4d2-df6936424cbc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-53f762b8-43ed-4289-a4d2-df6936424cbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:17:09.264313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:09.265280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:09.266297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:17:09.266297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:17:09.266297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:17:09.266297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:17:09.266297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:17:09.266297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:17:09.266297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:09.266297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:09.266297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 445/910] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:09.274042 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872104e-43a7-4be5-a0dd-ddb6ba73ce5d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:09.315874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c6d2d-0e28-41e6-b3d8-fcc1978ebfe5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:09.316209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c6d2d-0e28-41e6-b3d8-fcc1978ebfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:09.316432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c6d2d-0e28-41e6-b3d8-fcc1978ebfe5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:09.317877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 446/911] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:09.324314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a35b8-2e21-460d-a53c-909eb02c9c59 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:09.324836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a35b8-2e21-460d-a53c-909eb02c9c59 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:09.325294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a35b8-2e21-460d-a53c-909eb02c9c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:09.347774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7a3a35b8-2e21-460d-a53c-909eb02c9c59 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:09.349768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7a3a35b8-2e21-460d-a53c-909eb02c9c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:09.393769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872104e-43a7-4be5-a0dd-ddb6ba73ce5d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:09.434315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cba62e-b015-41a1-b809-aa22d230366f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:09.434919 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cba62e-b015-41a1-b809-aa22d230366f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:09.435193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cba62e-b015-41a1-b809-aa22d230366f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:09.435513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cba62e-b015-41a1-b809-aa22d230366f None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:09.436195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cba62e-b015-41a1-b809-aa22d230366f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:09.436662 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cba62e-b015-41a1-b809-aa22d230366f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:09.438177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cba62e-b015-41a1-b809-aa22d230366f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:09.452408 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 447/912] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:09.453814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 466/913] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:17:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:09.459782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a35b8-2e21-460d-a53c-909eb02c9c59 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:09.564514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a35b8-2e21-460d-a53c-909eb02c9c59 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:09.579375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:09.607179 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fecf88-44dc-4aa8-a3df-30b97068623f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:09.608140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fecf88-44dc-4aa8-a3df-30b97068623f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:09.608654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fecf88-44dc-4aa8-a3df-30b97068623f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:09.609122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fecf88-44dc-4aa8-a3df-30b97068623f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:09.610029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fecf88-44dc-4aa8-a3df-30b97068623f None admin] RBAC: Adding query filter params (role.id=657af95d19214896a42b23c30ab7a220, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=b638b8149ae149e3aad92ed51be92588) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:09.610549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fecf88-44dc-4aa8-a3df-30b97068623f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:09.612300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fecf88-44dc-4aa8-a3df-30b97068623f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:09.618345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 448/914] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:17:09 2022] GET /identity/v3/role_assignments?user.id=b638b8149ae149e3aad92ed51be92588&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=657af95d19214896a42b23c30ab7a220 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:09.743555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cba62e-b015-41a1-b809-aa22d230366f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:09.760660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:09.860331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cba62e-b015-41a1-b809-aa22d230366f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:09.897064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb73f7-8959-4222-87f5-82776a5a7d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:09.898174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb73f7-8959-4222-87f5-82776a5a7d14 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:09.898708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb73f7-8959-4222-87f5-82776a5a7d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:09.899188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb73f7-8959-4222-87f5-82776a5a7d14 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:09.899977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb73f7-8959-4222-87f5-82776a5a7d14 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:09.901566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb73f7-8959-4222-87f5-82776a5a7d14 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-efbb73f7-8959-4222-87f5-82776a5a7d14 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:09.914277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:09.915340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:09.916607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 467/915] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:17:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:09.923624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fecf88-44dc-4aa8-a3df-30b97068623f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:09.935657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d407d2-bc98-4aac-9f9e-f2dd23817a95 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:09.935932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d407d2-bc98-4aac-9f9e-f2dd23817a95 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:09.936203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d407d2-bc98-4aac-9f9e-f2dd23817a95 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:09.937657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 468/916] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:09.942607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43e8e2-66db-4070-92b8-c7285be8b62b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:09.943141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43e8e2-66db-4070-92b8-c7285be8b62b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:09.943851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43e8e2-66db-4070-92b8-c7285be8b62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:09.963921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4c43e8e2-66db-4070-92b8-c7285be8b62b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:09.965130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4c43e8e2-66db-4070-92b8-c7285be8b62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:10.035533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fecf88-44dc-4aa8-a3df-30b97068623f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:10.044390 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 469/917] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:10.086323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a41c6-c570-4fa2-98c2-d2569191a744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:10.086924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a41c6-c570-4fa2-98c2-d2569191a744 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:10.087202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a41c6-c570-4fa2-98c2-d2569191a744 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:10.087513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a41c6-c570-4fa2-98c2-d2569191a744 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:10.088192 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a41c6-c570-4fa2-98c2-d2569191a744 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:10.088525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a41c6-c570-4fa2-98c2-d2569191a744 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:10.089934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a41c6-c570-4fa2-98c2-d2569191a744 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:10.095225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 449/918] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:17:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:10.102061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43e8e2-66db-4070-92b8-c7285be8b62b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:10.120732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:10.209797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43e8e2-66db-4070-92b8-c7285be8b62b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:10.245647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a81b1-8b0c-4ad9-a29b-9a82bd931a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:10.246255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a81b1-8b0c-4ad9-a29b-9a82bd931a15 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:10.246527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a81b1-8b0c-4ad9-a29b-9a82bd931a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:10.246792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a81b1-8b0c-4ad9-a29b-9a82bd931a15 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:10.247415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a81b1-8b0c-4ad9-a29b-9a82bd931a15 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:10.247709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a81b1-8b0c-4ad9-a29b-9a82bd931a15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:10.249081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a81b1-8b0c-4ad9-a29b-9a82bd931a15 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:10.254460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 470/919] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Sep 30 19:17:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:10.261494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a41c6-c570-4fa2-98c2-d2569191a744 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:10.305897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:10.327846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a81b1-8b0c-4ad9-a29b-9a82bd931a15 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:10.368691 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a41c6-c570-4fa2-98c2-d2569191a744 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:10.416208 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029299e-23df-499b-beca-8eb0abd2ffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:10.416847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029299e-23df-499b-beca-8eb0abd2ffb7 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:10.417123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029299e-23df-499b-beca-8eb0abd2ffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:10.417391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029299e-23df-499b-beca-8eb0abd2ffb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:10.418059 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029299e-23df-499b-beca-8eb0abd2ffb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:10.419420 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029299e-23df-499b-beca-8eb0abd2ffb7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:10.421238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:10.452537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 450/920] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:17:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:17:10.474948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a81b1-8b0c-4ad9-a29b-9a82bd931a15 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:10.521732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5c7ba-5ecc-4d4a-9af1-ec57e7326da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:10.523306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5c7ba-5ecc-4d4a-9af1-ec57e7326da6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:10.523803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5c7ba-5ecc-4d4a-9af1-ec57e7326da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:10.524268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5c7ba-5ecc-4d4a-9af1-ec57e7326da6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:10.525116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b5c7ba-5ecc-4d4a-9af1-ec57e7326da6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:10.526707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b5c7ba-5ecc-4d4a-9af1-ec57e7326da6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f1b5c7ba-5ecc-4d4a-9af1-ec57e7326da6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:10.537982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:10.539322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:10.540181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:10.543283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 471/921] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 30 19:17:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:10.549929 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029299e-23df-499b-beca-8eb0abd2ffb7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:10.591447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0705cfc-6c7b-4a1e-a8a7-34c1f37f5131 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:10.591661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0705cfc-6c7b-4a1e-a8a7-34c1f37f5131 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:10.591946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0705cfc-6c7b-4a1e-a8a7-34c1f37f5131 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:10.593710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 472/922] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:10.599771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab0d4e-58ab-4659-8fc0-80f5389605ef None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:10.600126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab0d4e-58ab-4659-8fc0-80f5389605ef None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:10.600322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab0d4e-58ab-4659-8fc0-80f5389605ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:10.621150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7aab0d4e-58ab-4659-8fc0-80f5389605ef None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:10.633013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7aab0d4e-58ab-4659-8fc0-80f5389605ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:10.680310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:10.684227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029299e-23df-499b-beca-8eb0abd2ffb7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:10.728449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 473/923] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:10.731683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d24537-dd7a-4982-ba8c-46f22a58cc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:10.732291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d24537-dd7a-4982-ba8c-46f22a58cc2b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:10.732660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d24537-dd7a-4982-ba8c-46f22a58cc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:10.732927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d24537-dd7a-4982-ba8c-46f22a58cc2b None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:10.733649 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d24537-dd7a-4982-ba8c-46f22a58cc2b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:10.734004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d24537-dd7a-4982-ba8c-46f22a58cc2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:10.735374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d24537-dd7a-4982-ba8c-46f22a58cc2b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:10.745264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 451/924] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 30 19:17:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:10.751197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab0d4e-58ab-4659-8fc0-80f5389605ef None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:10.812299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:10.860053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab0d4e-58ab-4659-8fc0-80f5389605ef None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:10.894957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bf0b2-78ea-4b96-9624-507590015b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:10.895582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bf0b2-78ea-4b96-9624-507590015b22 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:10.895861 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bf0b2-78ea-4b96-9624-507590015b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:10.896133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bf0b2-78ea-4b96-9624-507590015b22 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:10.896791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bf0b2-78ea-4b96-9624-507590015b22 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:10.897091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bf0b2-78ea-4b96-9624-507590015b22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:10.898990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bf0b2-78ea-4b96-9624-507590015b22 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:10.904195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 474/925] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 30 19:17:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:10.910107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d24537-dd7a-4982-ba8c-46f22a58cc2b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:11.023883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d24537-dd7a-4982-ba8c-46f22a58cc2b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:11.025654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bf0b2-78ea-4b96-9624-507590015b22 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:11.065487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b49d20-48cc-4ddf-a57b-60cde80ff1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:11.066495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b49d20-48cc-4ddf-a57b-60cde80ff1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:11.067032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b49d20-48cc-4ddf-a57b-60cde80ff1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:11.067506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b49d20-48cc-4ddf-a57b-60cde80ff1f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:11.068593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b49d20-48cc-4ddf-a57b-60cde80ff1f3 None admin] RBAC: Adding query filter params (interface=public, service_id=edcbfb3293d442dcb5b537aa4a449951, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:11.069126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b49d20-48cc-4ddf-a57b-60cde80ff1f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:11.070898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b49d20-48cc-4ddf-a57b-60cde80ff1f3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:11.077297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 452/926] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 30 19:17:10 2022] GET /identity/v3/endpoints?service_id=edcbfb3293d442dcb5b537aa4a449951&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:11.083428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b49d20-48cc-4ddf-a57b-60cde80ff1f3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:11.115446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:11.129873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bf0b2-78ea-4b96-9624-507590015b22 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:11.153465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:11.173048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed786b-502a-40c9-8557-9785a6426680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:11.174569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed786b-502a-40c9-8557-9785a6426680 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:11.175085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed786b-502a-40c9-8557-9785a6426680 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:11.176003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed786b-502a-40c9-8557-9785a6426680 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:11.176834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed786b-502a-40c9-8557-9785a6426680 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:11.179695 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed786b-502a-40c9-8557-9785a6426680 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-9bed786b-502a-40c9-8557-9785a6426680 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:11.192331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:11.193420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 19:17:11.194431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:11.197096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 475/927] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Sep 30 19:17:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:11.202884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed786b-502a-40c9-8557-9785a6426680 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:11.207787 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b49d20-48cc-4ddf-a57b-60cde80ff1f3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:11.249173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9c7ea-13a9-4fe8-b98b-e343e7f3c838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:11.250071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c7ea-13a9-4fe8-b98b-e343e7f3c838 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:11.250569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c7ea-13a9-4fe8-b98b-e343e7f3c838 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:11.250979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c7ea-13a9-4fe8-b98b-e343e7f3c838 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:11.251864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9c7ea-13a9-4fe8-b98b-e343e7f3c838 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:11.253459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9c7ea-13a9-4fe8-b98b-e343e7f3c838 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:11.268727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 453/928] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 30 19:17:11 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:17:11.312191 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed786b-502a-40c9-8557-9785a6426680 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:11.350513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20227717-f1b8-485f-9ef3-2821c0221707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:11.351709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20227717-f1b8-485f-9ef3-2821c0221707 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:11.351990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20227717-f1b8-485f-9ef3-2821c0221707 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:11.352261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20227717-f1b8-485f-9ef3-2821c0221707 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:11.352938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20227717-f1b8-485f-9ef3-2821c0221707 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:11.353239 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20227717-f1b8-485f-9ef3-2821c0221707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:11.354618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20227717-f1b8-485f-9ef3-2821c0221707 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:11.360356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 476/929] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 30 19:17:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:11.366444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9c7ea-13a9-4fe8-b98b-e343e7f3c838 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:11.485262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:11.496968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9c7ea-13a9-4fe8-b98b-e343e7f3c838 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:11.533223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a75f36-9125-411b-8afa-6a8af3cd4559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:11.533850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a75f36-9125-411b-8afa-6a8af3cd4559 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:11.534156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a75f36-9125-411b-8afa-6a8af3cd4559 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:11.534429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a75f36-9125-411b-8afa-6a8af3cd4559 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:11.535060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a75f36-9125-411b-8afa-6a8af3cd4559 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:11.535400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a75f36-9125-411b-8afa-6a8af3cd4559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:11.536847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a75f36-9125-411b-8afa-6a8af3cd4559 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:11.542603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 454/930] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 30 19:17:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:11.548277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20227717-f1b8-485f-9ef3-2821c0221707 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:11.604715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:11.654674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20227717-f1b8-485f-9ef3-2821c0221707 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:11.690909 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff305e52-ec1a-4f58-aa86-12476fb2b114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:11.691584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff305e52-ec1a-4f58-aa86-12476fb2b114 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:11.691862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff305e52-ec1a-4f58-aa86-12476fb2b114 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:11.692135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff305e52-ec1a-4f58-aa86-12476fb2b114 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:11.692902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff305e52-ec1a-4f58-aa86-12476fb2b114 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:11.694327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff305e52-ec1a-4f58-aa86-12476fb2b114 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:11.716983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 477/931] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:17:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:17:11.954544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5ff35-43fb-427f-a31a-de075677c7d6 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:11.954936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5ff35-43fb-427f-a31a-de075677c7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:11.955120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5ff35-43fb-427f-a31a-de075677c7d6 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:11.956887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 455/932] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:11.962143 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04c8b-1c27-4592-8728-8366ad3bd47a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:11.962434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04c8b-1c27-4592-8728-8366ad3bd47a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:11.962713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04c8b-1c27-4592-8728-8366ad3bd47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:11.982368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bfd04c8b-1c27-4592-8728-8366ad3bd47a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:11.983553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bfd04c8b-1c27-4592-8728-8366ad3bd47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:12.058873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 478/933] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:12.328274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5ff35-43fb-427f-a31a-de075677c7d6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:12.450282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5ff35-43fb-427f-a31a-de075677c7d6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:12.486763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfffe5-6517-4f3e-b3f7-4eea0393dd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:12.487404 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfffe5-6517-4f3e-b3f7-4eea0393dd10 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:12.487679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfffe5-6517-4f3e-b3f7-4eea0393dd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:12.487946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfffe5-6517-4f3e-b3f7-4eea0393dd10 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:12.499195 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfffe5-6517-4f3e-b3f7-4eea0393dd10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-28dfffe5-6517-4f3e-b3f7-4eea0393dd10 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:17:12.503976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:12.504948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:12.505813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 456/934] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 30 19:17:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:12.511101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd04c8b-1c27-4592-8728-8366ad3bd47a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:12.546069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:12.615827 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd04c8b-1c27-4592-8728-8366ad3bd47a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:12.655070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f3f7e-cb06-4df7-83a4-6b8f3a4be33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:12.655706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f3f7e-cb06-4df7-83a4-6b8f3a4be33f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:12.655983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f3f7e-cb06-4df7-83a4-6b8f3a4be33f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:12.656253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f3f7e-cb06-4df7-83a4-6b8f3a4be33f None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:12.656973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f3f7e-cb06-4df7-83a4-6b8f3a4be33f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:12.657274 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f3f7e-cb06-4df7-83a4-6b8f3a4be33f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:12.658688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f3f7e-cb06-4df7-83a4-6b8f3a4be33f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:12.669788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 479/935] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 30 19:17:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:12.675673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfffe5-6517-4f3e-b3f7-4eea0393dd10 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:12.692547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:12.772518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29021fe3-a7fa-415d-8dc9-2b0fe6c9d424 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:12.772891 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29021fe3-a7fa-415d-8dc9-2b0fe6c9d424 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:12.773095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29021fe3-a7fa-415d-8dc9-2b0fe6c9d424 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:12.774651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 480/936] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:12.780316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfffe5-6517-4f3e-b3f7-4eea0393dd10 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:12.781197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b1709-402f-4970-9363-0c95dc1f2fcf None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:12.781588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b1709-402f-4970-9363-0c95dc1f2fcf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:12.782107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b1709-402f-4970-9363-0c95dc1f2fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:12.803497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5d5b1709-402f-4970-9363-0c95dc1f2fcf None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:12.805373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5d5b1709-402f-4970-9363-0c95dc1f2fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:12.816057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d3fe7-65ce-4282-9ed2-de240ffc97bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:12.816765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d3fe7-65ce-4282-9ed2-de240ffc97bd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:12.817049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d3fe7-65ce-4282-9ed2-de240ffc97bd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:12.817326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d3fe7-65ce-4282-9ed2-de240ffc97bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:12.830143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d3fe7-65ce-4282-9ed2-de240ffc97bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:12.832101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d3fe7-65ce-4282-9ed2-de240ffc97bd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-a44d3fe7-65ce-4282-9ed2-de240ffc97bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:12.848594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:12.849743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:12.850922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:12.850922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:12.850922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:12.856809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 457/937] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:12.857721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d3fe7-65ce-4282-9ed2-de240ffc97bd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:12.876357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 481/938] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:12.931081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:12.970339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d3fe7-65ce-4282-9ed2-de240ffc97bd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:13.006341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6955b5e-3942-46c6-9404-a82539f973cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:13.007295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6955b5e-3942-46c6-9404-a82539f973cf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.007767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6955b5e-3942-46c6-9404-a82539f973cf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.008260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6955b5e-3942-46c6-9404-a82539f973cf None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.009094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6955b5e-3942-46c6-9404-a82539f973cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:13.009565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6955b5e-3942-46c6-9404-a82539f973cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:13.011101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6955b5e-3942-46c6-9404-a82539f973cf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:13.023860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 458/939] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:13.029602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b1709-402f-4970-9363-0c95dc1f2fcf None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:13.057995 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:13.133083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b1709-402f-4970-9363-0c95dc1f2fcf None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:13.172755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb34223-bb25-45c9-8434-a56200bfc0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:13.173386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb34223-bb25-45c9-8434-a56200bfc0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.173660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb34223-bb25-45c9-8434-a56200bfc0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.173928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb34223-bb25-45c9-8434-a56200bfc0f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.181166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb34223-bb25-45c9-8434-a56200bfc0f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:13.182693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb34223-bb25-45c9-8434-a56200bfc0f9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:13.183704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6955b5e-3942-46c6-9404-a82539f973cf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-adb34223-bb25-45c9-8434-a56200bfc0f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:17:13.194395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:13.195934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:13.196972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:13.196972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:13.196972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:17:13.196972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:17:13.196972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:17:13.196972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:17:13.196972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:17:13.196972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:17:13.196972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:17:13.196972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:13.196972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 482/940] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 30 19:17:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:13.201822 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb34223-bb25-45c9-8434-a56200bfc0f9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:13.288462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:13.301558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6955b5e-3942-46c6-9404-a82539f973cf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:13.303568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb34223-bb25-45c9-8434-a56200bfc0f9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:13.330081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:13.338425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391689e0-7109-4b74-bd7d-816f432a5df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:13.339051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391689e0-7109-4b74-bd7d-816f432a5df6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.339337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391689e0-7109-4b74-bd7d-816f432a5df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.339595 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391689e0-7109-4b74-bd7d-816f432a5df6 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.340321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391689e0-7109-4b74-bd7d-816f432a5df6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:13.340652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391689e0-7109-4b74-bd7d-816f432a5df6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:13.342145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391689e0-7109-4b74-bd7d-816f432a5df6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:13.343353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57321a-881c-437c-8c34-acd6485bdeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:13.344272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57321a-881c-437c-8c34-acd6485bdeeb None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.344910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57321a-881c-437c-8c34-acd6485bdeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.345390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57321a-881c-437c-8c34-acd6485bdeeb None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.346360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57321a-881c-437c-8c34-acd6485bdeeb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:13.348039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57321a-881c-437c-8c34-acd6485bdeeb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:13.351677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: WARNING keystone.server.flask.common [None req-6e57321a-881c-437c-8c34-acd6485bdeeb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 30 19:17:13.368597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 483/941] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 30 19:17:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:13.368597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391689e0-7109-4b74-bd7d-816f432a5df6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:13.371907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.sql.core [None req-6e57321a-881c-437c-8c34-acd6485bdeeb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 30 19:17:13.371907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 30 19:17:13.371907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [parameters: {'id': '7d4b4450152246cba770d4854e3dff1d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 30 19:17:13.371907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81403) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6e57321a-881c-437c-8c34-acd6485bdeeb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7d4b4450152246cba770d4854e3dff1d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7d4b4450152246cba770d4854e3dff1d. Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 19:17:13.379287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 30 19:17:13.380154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application yield resource Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application next(self.gen) Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.rollback() Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.force_reraise() Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise self.value Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application session.commit() Sep 30 19:17:13.381507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._prepare_impl() Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.session.flush() Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._flush(objects) Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application compat.raise_( Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application flush_context.execute() Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 30 19:17:13.382854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rec.execute(self) Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 30 19:17:13.384012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application conn.query(q) Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application result.read() Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 19:17:13.386777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [parameters: {'id': '7d4b4450152246cba770d4854e3dff1d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:13.388982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:13.390995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:13.393442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:13.393442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 30 19:17:13.393442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 30 19:17:13.393442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:13.393442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:13.393442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 30 19:17:13.393442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 30 19:17:13.393442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7d4b4450152246cba770d4854e3dff1d. Sep 30 19:17:13.393442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:13.393442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 459/942] 10.222.0.17 () {60 vars in 1140 bytes} [Fri Sep 30 19:17:13 2022] POST /identity/v3/projects => generated 262 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:13.393442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57321a-881c-437c-8c34-acd6485bdeeb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:13.490183 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391689e0-7109-4b74-bd7d-816f432a5df6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:13.493903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57321a-881c-437c-8c34-acd6485bdeeb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:13.530325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e066ec-bda3-46b1-8502-54ac421df30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:13.531369 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e066ec-bda3-46b1-8502-54ac421df30a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.531696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc532ed-693f-41ae-9277-3f1b6a182361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:13.532112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e066ec-bda3-46b1-8502-54ac421df30a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.532336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc532ed-693f-41ae-9277-3f1b6a182361 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.532662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc532ed-693f-41ae-9277-3f1b6a182361 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.532932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc532ed-693f-41ae-9277-3f1b6a182361 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.533241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e066ec-bda3-46b1-8502-54ac421df30a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.541138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc532ed-693f-41ae-9277-3f1b6a182361 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:13.542807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc532ed-693f-41ae-9277-3f1b6a182361 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:13.546834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e066ec-bda3-46b1-8502-54ac421df30a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:13.548614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e066ec-bda3-46b1-8502-54ac421df30a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-efc532ed-693f-41ae-9277-3f1b6a182361 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:17:13.555821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:13.556790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:13.558113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:17:13.558113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:17:13.558113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:17:13.558113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:17:13.558113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:17:13.558113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:17:13.558113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:13.558113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:13.558113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 460/943] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:13.561905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:13.565466 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc532ed-693f-41ae-9277-3f1b6a182361 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f2e066ec-bda3-46b1-8502-54ac421df30a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:13.572951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:13.573940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:13.574872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:13.574872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:13.574872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:13.583653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 484/944] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:13.586594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e066ec-bda3-46b1-8502-54ac421df30a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:13.617840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:13.682591 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc532ed-693f-41ae-9277-3f1b6a182361 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:13.694535 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e066ec-bda3-46b1-8502-54ac421df30a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:13.716471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c8ddb-b6ac-4129-8d48-20dea605e820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:13.717077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c8ddb-b6ac-4129-8d48-20dea605e820 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.717354 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c8ddb-b6ac-4129-8d48-20dea605e820 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.717622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c8ddb-b6ac-4129-8d48-20dea605e820 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.718285 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c8ddb-b6ac-4129-8d48-20dea605e820 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:13.718583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c8ddb-b6ac-4129-8d48-20dea605e820 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:13.720009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c8ddb-b6ac-4129-8d48-20dea605e820 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:13.732173 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14798eb2-e88e-4a36-88d0-58d6b3e7c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:13.733048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14798eb2-e88e-4a36-88d0-58d6b3e7c34d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.733643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14798eb2-e88e-4a36-88d0-58d6b3e7c34d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.733915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14798eb2-e88e-4a36-88d0-58d6b3e7c34d None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.734551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14798eb2-e88e-4a36-88d0-58d6b3e7c34d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:13.734855 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14798eb2-e88e-4a36-88d0-58d6b3e7c34d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:13.736247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14798eb2-e88e-4a36-88d0-58d6b3e7c34d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:13.746622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 461/945] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:17:13 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:13.746622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01270b71-72c2-4af9-8637-7be5d2b9210e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.746622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01270b71-72c2-4af9-8637-7be5d2b9210e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.746622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01270b71-72c2-4af9-8637-7be5d2b9210e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.746622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 462/946] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:13.749052 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47bc6c-8f49-4710-90f3-1c81dcede32e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.749340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47bc6c-8f49-4710-90f3-1c81dcede32e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.749614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47bc6c-8f49-4710-90f3-1c81dcede32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.764745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 485/947] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:13.766257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14798eb2-e88e-4a36-88d0-58d6b3e7c34d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:13.770303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0d47bc6c-8f49-4710-90f3-1c81dcede32e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:13.771980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0d47bc6c-8f49-4710-90f3-1c81dcede32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:13.836644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:13.881170 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 463/948] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:13.894208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14798eb2-e88e-4a36-88d0-58d6b3e7c34d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:13.932817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43282f4-fab8-45a1-b79c-c870a3c1e8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:13.933389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43282f4-fab8-45a1-b79c-c870a3c1e8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.933664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43282f4-fab8-45a1-b79c-c870a3c1e8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.933935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43282f4-fab8-45a1-b79c-c870a3c1e8c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.941629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43282f4-fab8-45a1-b79c-c870a3c1e8c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:13.943135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43282f4-fab8-45a1-b79c-c870a3c1e8c3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-b43282f4-fab8-45a1-b79c-c870a3c1e8c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:17:13.956091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:13.957084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:13.958400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:17:13.958400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:17:13.958400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:17:13.958400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:17:13.958400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:17:13.958400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:17:13.958400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:13.958400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:13.958400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 486/949] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:13.964458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47bc6c-8f49-4710-90f3-1c81dcede32e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:13.968811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7526df-b683-4597-b43e-38ace1acf5cf None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.969091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7526df-b683-4597-b43e-38ace1acf5cf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.969362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7526df-b683-4597-b43e-38ace1acf5cf None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.970991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 487/950] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:13.976165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249bde1-54f8-44c1-bb91-8d0cce336145 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:13.976487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249bde1-54f8-44c1-bb91-8d0cce336145 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:13.976763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249bde1-54f8-44c1-bb91-8d0cce336145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:13.998877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4249bde1-54f8-44c1-bb91-8d0cce336145 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:14.000185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4249bde1-54f8-44c1-bb91-8d0cce336145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:14.003188 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:14.088787 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 488/951] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:14.094342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47bc6c-8f49-4710-90f3-1c81dcede32e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:14.131863 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5e777-32d5-444c-b6a8-2dd2721ab164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:14.132579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5e777-32d5-444c-b6a8-2dd2721ab164 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:14.132963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5e777-32d5-444c-b6a8-2dd2721ab164 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:14.133439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5e777-32d5-444c-b6a8-2dd2721ab164 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:14.134155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5e777-32d5-444c-b6a8-2dd2721ab164 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:14.134456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5e777-32d5-444c-b6a8-2dd2721ab164 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:14.135934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5e777-32d5-444c-b6a8-2dd2721ab164 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:14.148279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 464/952] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:17:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:14.154194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249bde1-54f8-44c1-bb91-8d0cce336145 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:14.190904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5e777-32d5-444c-b6a8-2dd2721ab164 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:14.202241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:14.259900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249bde1-54f8-44c1-bb91-8d0cce336145 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:14.300673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c395ae3-244d-4ae1-9482-ba195e38a22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:14.301339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c395ae3-244d-4ae1-9482-ba195e38a22e None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:14.301616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c395ae3-244d-4ae1-9482-ba195e38a22e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:14.301887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c395ae3-244d-4ae1-9482-ba195e38a22e None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/b638b8149ae149e3aad92ed51be92588/roles/657af95d19214896a42b23c30ab7a220` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:14.305432 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c395ae3-244d-4ae1-9482-ba195e38a22e None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=b638b8149ae149e3aad92ed51be92588, role_id=657af95d19214896a42b23c30ab7a220)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:14.307008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:14.308939 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c395ae3-244d-4ae1-9482-ba195e38a22e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:14.316060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5e777-32d5-444c-b6a8-2dd2721ab164 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:14.327981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 489/953] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:17:14 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/b638b8149ae149e3aad92ed51be92588/roles/657af95d19214896a42b23c30ab7a220 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:17:14.363174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aee9bc-a09c-47f9-ab38-05e44b5ff108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:14.363881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aee9bc-a09c-47f9-ab38-05e44b5ff108 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:14.364214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aee9bc-a09c-47f9-ab38-05e44b5ff108 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:14.364561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aee9bc-a09c-47f9-ab38-05e44b5ff108 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:14.365156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aee9bc-a09c-47f9-ab38-05e44b5ff108 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:14.367138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aee9bc-a09c-47f9-ab38-05e44b5ff108 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-64aee9bc-a09c-47f9-ab38-05e44b5ff108 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:14.377952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:14.379451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:14.380637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:14.382905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c395ae3-244d-4ae1-9482-ba195e38a22e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:14.384464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 465/954] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 30 19:17:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:14.386299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aee9bc-a09c-47f9-ab38-05e44b5ff108 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:14.528764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c395ae3-244d-4ae1-9482-ba195e38a22e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:14.536798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:14.536968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aee9bc-a09c-47f9-ab38-05e44b5ff108 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:14.570749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a8f65-8492-4739-bfcc-62f580921b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:14.571538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a8f65-8492-4739-bfcc-62f580921b5a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:14.571905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a8f65-8492-4739-bfcc-62f580921b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:14.572580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a8f65-8492-4739-bfcc-62f580921b5a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:14.573019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a8f65-8492-4739-bfcc-62f580921b5a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:14.574455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a8f65-8492-4739-bfcc-62f580921b5a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-ce8a8f65-8492-4739-bfcc-62f580921b5a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:14.583493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:14.584771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 19:17:14.585915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:14.587785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 490/955] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:14.588069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efefc3-6f7a-4ca4-97d3-8705cf89660a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:14.588907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efefc3-6f7a-4ca4-97d3-8705cf89660a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:14.589304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efefc3-6f7a-4ca4-97d3-8705cf89660a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:14.589713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efefc3-6f7a-4ca4-97d3-8705cf89660a None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:14.590471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efefc3-6f7a-4ca4-97d3-8705cf89660a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:14.590903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efefc3-6f7a-4ca4-97d3-8705cf89660a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:14.592533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efefc3-6f7a-4ca4-97d3-8705cf89660a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:14.593572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a8f65-8492-4739-bfcc-62f580921b5a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:14.599379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 466/956] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 30 19:17:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:14.609348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efefc3-6f7a-4ca4-97d3-8705cf89660a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:14.641957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:14.714182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a8f65-8492-4739-bfcc-62f580921b5a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:14.744873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efefc3-6f7a-4ca4-97d3-8705cf89660a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:14.759535 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197c72a-7283-4343-972f-f661e35993cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:14.760149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197c72a-7283-4343-972f-f661e35993cb None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:14.760900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197c72a-7283-4343-972f-f661e35993cb None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:14.761177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197c72a-7283-4343-972f-f661e35993cb None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:14.761799 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197c72a-7283-4343-972f-f661e35993cb None admin] RBAC: Adding query filter params (name=compute) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:14.762369 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197c72a-7283-4343-972f-f661e35993cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:14.763975 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197c72a-7283-4343-972f-f661e35993cb None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:14.770022 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 491/957] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:17:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:14.775876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197c72a-7283-4343-972f-f661e35993cb None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:14.785662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bd94a-634f-4751-b488-6533f901f48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:14.786592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bd94a-634f-4751-b488-6533f901f48a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:14.787017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bd94a-634f-4751-b488-6533f901f48a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:14.787531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bd94a-634f-4751-b488-6533f901f48a None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:14.788818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bd94a-634f-4751-b488-6533f901f48a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:14.789287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bd94a-634f-4751-b488-6533f901f48a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:14.790929 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bd94a-634f-4751-b488-6533f901f48a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:14.802163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 467/958] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 30 19:17:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:14.803851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bd94a-634f-4751-b488-6533f901f48a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:14.814187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:14.884637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197c72a-7283-4343-972f-f661e35993cb None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:14.899247 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:14.918560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bd94a-634f-4751-b488-6533f901f48a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:14.918832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfeb3ff-6b97-48a5-9397-9dd796fbcebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:14.919438 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfeb3ff-6b97-48a5-9397-9dd796fbcebc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:14.919713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfeb3ff-6b97-48a5-9397-9dd796fbcebc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:14.919981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfeb3ff-6b97-48a5-9397-9dd796fbcebc None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:14.920642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfeb3ff-6b97-48a5-9397-9dd796fbcebc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:14.920938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfeb3ff-6b97-48a5-9397-9dd796fbcebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:14.922290 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfeb3ff-6b97-48a5-9397-9dd796fbcebc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:14.931633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 492/959] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:17:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:14.968851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86c20b-ab1d-4c98-9861-f3f710787ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:14.969861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86c20b-ab1d-4c98-9861-f3f710787ea2 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:14.970344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86c20b-ab1d-4c98-9861-f3f710787ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:14.970810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86c20b-ab1d-4c98-9861-f3f710787ea2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:14.971753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86c20b-ab1d-4c98-9861-f3f710787ea2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:14.973421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86c20b-ab1d-4c98-9861-f3f710787ea2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:15.004631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 468/960] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:17:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:17:16.107523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb079e6-9876-442f-ad29-f168eb2ceb6e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:16.108051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb079e6-9876-442f-ad29-f168eb2ceb6e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:16.108051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb079e6-9876-442f-ad29-f168eb2ceb6e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:16.109679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 493/961] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:16.115079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66663ba-42f5-4055-8576-5172a54d3c78 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:16.115492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66663ba-42f5-4055-8576-5172a54d3c78 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:16.115675 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66663ba-42f5-4055-8576-5172a54d3c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:16.134309 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a66663ba-42f5-4055-8576-5172a54d3c78 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:16.135471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a66663ba-42f5-4055-8576-5172a54d3c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:16.216030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 469/962] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:16.498094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb079e6-9876-442f-ad29-f168eb2ceb6e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:16.553008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027adf8-2b78-43a4-9e6b-042274ce627c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:16.553324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027adf8-2b78-43a4-9e6b-042274ce627c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:16.553600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027adf8-2b78-43a4-9e6b-042274ce627c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:16.555715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 470/963] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:16.560759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bccd12-5db9-43a7-855c-d51c204fbdba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:16.561039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bccd12-5db9-43a7-855c-d51c204fbdba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:16.561311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bccd12-5db9-43a7-855c-d51c204fbdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:16.582747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-97bccd12-5db9-43a7-855c-d51c204fbdba None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:16.584617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-97bccd12-5db9-43a7-855c-d51c204fbdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:16.602146 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:16.608800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb079e6-9876-442f-ad29-f168eb2ceb6e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:16.648090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746030f2-7ce4-4b08-a2b5-c1f83b3afdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:16.649019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746030f2-7ce4-4b08-a2b5-c1f83b3afdb7 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:16.649300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746030f2-7ce4-4b08-a2b5-c1f83b3afdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:16.649567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746030f2-7ce4-4b08-a2b5-c1f83b3afdb7 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:16.650194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746030f2-7ce4-4b08-a2b5-c1f83b3afdb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:16.652113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746030f2-7ce4-4b08-a2b5-c1f83b3afdb7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:16.654894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: WARNING keystone.server.flask.common [None req-746030f2-7ce4-4b08-a2b5-c1f83b3afdb7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 30 19:17:16.658401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:16.668547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.sql.core [None req-746030f2-7ce4-4b08-a2b5-c1f83b3afdb7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 30 19:17:16.668547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 30 19:17:16.668547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [parameters: {'user_id': 'afb506c45fe64ab8aed9bec44f6338c7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 30 19:17:16.668547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81402) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 30 19:17:16.680390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 471/964] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-746030f2-7ce4-4b08-a2b5-c1f83b3afdb7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application conn.query(q) Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application result.read() Sep 30 19:17:16.690748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 19:17:16.691677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application next(self.gen) Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application yield resource Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application next(self.gen) Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self.session.rollback() Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self.force_reraise() Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise self.value Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application session.commit() Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 30 19:17:16.692721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._prepare_impl() Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self.session.flush() Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._flush(objects) Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application compat.raise_( Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application flush_context.execute() Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rec.execute(self) Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 30 19:17:16.693655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 19:17:16.694673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application conn.query(q) Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application result.read() Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 19:17:16.695582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [parameters: {'user_id': 'afb506c45fe64ab8aed9bec44f6338c7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:16.698280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 30 19:17:16.700354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 30 19:17:16.707608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 30 19:17:16.707608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 30 19:17:16.707608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 30 19:17:16.707608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 30 19:17:16.707608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 30 19:17:16.707608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:16.707608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 494/965] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Sep 30 19:17:16 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:16.708682 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bccd12-5db9-43a7-855c-d51c204fbdba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:16.809809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bccd12-5db9-43a7-855c-d51c204fbdba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:16.846061 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cd4c6-7aff-4adc-b54e-6ecff7c088e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:16.846691 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cd4c6-7aff-4adc-b54e-6ecff7c088e3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:16.846966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cd4c6-7aff-4adc-b54e-6ecff7c088e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:16.847236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cd4c6-7aff-4adc-b54e-6ecff7c088e3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:16.855191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cd4c6-7aff-4adc-b54e-6ecff7c088e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:16.856653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cd4c6-7aff-4adc-b54e-6ecff7c088e3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9a6cd4c6-7aff-4adc-b54e-6ecff7c088e3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:17:16.869276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:16.871479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:16.873913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:16.873913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:16.873913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:17:16.873913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:17:16.873913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:17:16.873913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:17:16.873913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:17:16.873913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:17:16.873913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 19:17:16.873913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:16.873913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 472/966] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:17:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:16.880412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746030f2-7ce4-4b08-a2b5-c1f83b3afdb7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:16.972745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cd4c6-7aff-4adc-b54e-6ecff7c088e3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:16.987072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746030f2-7ce4-4b08-a2b5-c1f83b3afdb7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:17.030897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4839e2-4951-4536-9065-0db252163aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:17.031669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4839e2-4951-4536-9065-0db252163aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.032104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4839e2-4951-4536-9065-0db252163aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.032606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4839e2-4951-4536-9065-0db252163aa8 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.033378 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4839e2-4951-4536-9065-0db252163aa8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:17.033821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4839e2-4951-4536-9065-0db252163aa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:17.035533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4839e2-4951-4536-9065-0db252163aa8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:17.047265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:17.088663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 495/967] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Sep 30 19:17:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:17.110180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b748ee1-7de3-4701-add1-54bb3e738a37 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.110836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b748ee1-7de3-4701-add1-54bb3e738a37 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.110836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b748ee1-7de3-4701-add1-54bb3e738a37 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.112488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 496/968] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:17.113433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cd4c6-7aff-4adc-b54e-6ecff7c088e3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:17.120990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34014b1-e593-4c6d-8dba-9f6c52f87b96 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.121607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34014b1-e593-4c6d-8dba-9f6c52f87b96 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.122152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34014b1-e593-4c6d-8dba-9f6c52f87b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.144856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d34014b1-e593-4c6d-8dba-9f6c52f87b96 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:17.146928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d34014b1-e593-4c6d-8dba-9f6c52f87b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:17.157781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad6a35-d2db-4903-b57a-c04a2b6b292a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:17.158505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad6a35-d2db-4903-b57a-c04a2b6b292a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.158711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad6a35-d2db-4903-b57a-c04a2b6b292a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.158977 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad6a35-d2db-4903-b57a-c04a2b6b292a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.171848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad6a35-d2db-4903-b57a-c04a2b6b292a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:17:17.173631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-23ad6a35-d2db-4903-b57a-c04a2b6b292a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:17:17.177289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:17.178336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:17.179280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 473/969] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 30 19:17:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:17.186490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad6a35-d2db-4903-b57a-c04a2b6b292a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:17.266202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 497/970] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:17.267636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a627d-4965-4c9c-a4db-aea2e4dc2ea3 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.267920 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a627d-4965-4c9c-a4db-aea2e4dc2ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.268194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a627d-4965-4c9c-a4db-aea2e4dc2ea3 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.269701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 498/971] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:17.276169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529871c2-e2b1-4faf-82a7-79e7c545d90f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.276604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529871c2-e2b1-4faf-82a7-79e7c545d90f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.276998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529871c2-e2b1-4faf-82a7-79e7c545d90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.298031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-529871c2-e2b1-4faf-82a7-79e7c545d90f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:17.299333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-529871c2-e2b1-4faf-82a7-79e7c545d90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:17.309955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad6a35-d2db-4903-b57a-c04a2b6b292a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:17.331477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:17.348950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee41466-9c36-4996-94e9-5c43394231d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:17.349556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee41466-9c36-4996-94e9-5c43394231d0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.349835 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee41466-9c36-4996-94e9-5c43394231d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.350104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee41466-9c36-4996-94e9-5c43394231d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.352549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee41466-9c36-4996-94e9-5c43394231d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:17.352549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee41466-9c36-4996-94e9-5c43394231d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:17.352549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee41466-9c36-4996-94e9-5c43394231d0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:17.362894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 474/972] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 30 19:17:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:17.370152 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee41466-9c36-4996-94e9-5c43394231d0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:17.379211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 499/973] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:17.488601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee41466-9c36-4996-94e9-5c43394231d0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:17.522764 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06490f-a516-439b-96e9-9b243968f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:17.523315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06490f-a516-439b-96e9-9b243968f00b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.523586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06490f-a516-439b-96e9-9b243968f00b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.523850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06490f-a516-439b-96e9-9b243968f00b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.535155 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529871c2-e2b1-4faf-82a7-79e7c545d90f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:17.535293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06490f-a516-439b-96e9-9b243968f00b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:17.536653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06490f-a516-439b-96e9-9b243968f00b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ca06490f-a516-439b-96e9-9b243968f00b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:17.551706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:17.552703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:17.553637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:17.553637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:17.553637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:17.553637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 475/974] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:17.558827 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06490f-a516-439b-96e9-9b243968f00b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:17.589521 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:17.618160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:17.640302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529871c2-e2b1-4faf-82a7-79e7c545d90f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:17.667283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06490f-a516-439b-96e9-9b243968f00b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:17.674592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb9f4-d7f6-456a-808b-78b275a6e8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:17.675194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb9f4-d7f6-456a-808b-78b275a6e8e3 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.675469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb9f4-d7f6-456a-808b-78b275a6e8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.675740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb9f4-d7f6-456a-808b-78b275a6e8e3 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.676399 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cb9f4-d7f6-456a-808b-78b275a6e8e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:17.677759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cb9f4-d7f6-456a-808b-78b275a6e8e3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:17.691891 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 500/975] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:17:17 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:17:17.692703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb9f4-d7f6-456a-808b-78b275a6e8e3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:17.714980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b16ad5-c33e-421a-9473-9d8671cb4bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:17.715609 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b16ad5-c33e-421a-9473-9d8671cb4bfa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.715885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b16ad5-c33e-421a-9473-9d8671cb4bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.716153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b16ad5-c33e-421a-9473-9d8671cb4bfa None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.716854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b16ad5-c33e-421a-9473-9d8671cb4bfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:17.717313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b16ad5-c33e-421a-9473-9d8671cb4bfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:17.718736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b16ad5-c33e-421a-9473-9d8671cb4bfa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:17.732276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 476/976] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:17.737990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b16ad5-c33e-421a-9473-9d8671cb4bfa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:17.813459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb9f4-d7f6-456a-808b-78b275a6e8e3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:17.829466 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:17.846176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b16ad5-c33e-421a-9473-9d8671cb4bfa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:17.856106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26000121-650d-4840-be5b-740fc190314d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:17.856771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26000121-650d-4840-be5b-740fc190314d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.857347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26000121-650d-4840-be5b-740fc190314d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.857347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26000121-650d-4840-be5b-740fc190314d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.857917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26000121-650d-4840-be5b-740fc190314d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:17.859282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26000121-650d-4840-be5b-740fc190314d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-26000121-650d-4840-be5b-740fc190314d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:17.868943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:17.870010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:17.870894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 501/977] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 30 19:17:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:17.876429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26000121-650d-4840-be5b-740fc190314d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:17.879766 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:17.889219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4e014-a2e3-49a9-9599-aa5d5a74011f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:17.889850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4e014-a2e3-49a9-9599-aa5d5a74011f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:17.890127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4e014-a2e3-49a9-9599-aa5d5a74011f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:17.890400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4e014-a2e3-49a9-9599-aa5d5a74011f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:17.901235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4e014-a2e3-49a9-9599-aa5d5a74011f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:17.903325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4e014-a2e3-49a9-9599-aa5d5a74011f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-fff4e014-a2e3-49a9-9599-aa5d5a74011f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:17:17.915089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:17.916089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:17.918374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:17.918374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:17.918374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:17:17.918374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:17:17.918374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:17:17.918374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:17:17.918374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:17:17.918374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:17:17.918374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 19:17:17.918374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:17.918374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 477/978] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 30 19:17:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:17.921874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4e014-a2e3-49a9-9599-aa5d5a74011f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:17.991022 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26000121-650d-4840-be5b-740fc190314d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:18.026428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4e014-a2e3-49a9-9599-aa5d5a74011f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:18.029757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251ce43-182b-4923-a147-2785f2a394af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:18.030367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251ce43-182b-4923-a147-2785f2a394af None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:18.030645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251ce43-182b-4923-a147-2785f2a394af None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:18.030914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251ce43-182b-4923-a147-2785f2a394af None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:18.031534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251ce43-182b-4923-a147-2785f2a394af None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:18.031832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251ce43-182b-4923-a147-2785f2a394af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:18.033214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251ce43-182b-4923-a147-2785f2a394af None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:18.042914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 502/979] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 30 19:17:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:18.043975 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251ce43-182b-4923-a147-2785f2a394af None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:18.065989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a18c7-8ac2-4c98-a6a4-b9f17dc01efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:18.066520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a18c7-8ac2-4c98-a6a4-b9f17dc01efa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:18.066796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a18c7-8ac2-4c98-a6a4-b9f17dc01efa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:18.067061 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a18c7-8ac2-4c98-a6a4-b9f17dc01efa None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:18.067785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a18c7-8ac2-4c98-a6a4-b9f17dc01efa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:18.068081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a18c7-8ac2-4c98-a6a4-b9f17dc01efa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:18.070098 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a18c7-8ac2-4c98-a6a4-b9f17dc01efa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:18.092583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 478/980] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Sep 30 19:17:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:18.093355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a18c7-8ac2-4c98-a6a4-b9f17dc01efa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:18.097028 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:18.112597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:18.149687 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251ce43-182b-4923-a147-2785f2a394af None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:18.186867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128a824-1ae0-41b6-be36-d7e41efe7d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:18.187461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128a824-1ae0-41b6-be36-d7e41efe7d81 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:18.187773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128a824-1ae0-41b6-be36-d7e41efe7d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:18.188053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128a824-1ae0-41b6-be36-d7e41efe7d81 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:18.188645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128a824-1ae0-41b6-be36-d7e41efe7d81 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:18.188938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128a824-1ae0-41b6-be36-d7e41efe7d81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:18.190290 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128a824-1ae0-41b6-be36-d7e41efe7d81 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:18.196543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 503/981] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 30 19:17:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:18.199303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a18c7-8ac2-4c98-a6a4-b9f17dc01efa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:18.202204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128a824-1ae0-41b6-be36-d7e41efe7d81 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:18.234399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6199ec-4dbf-46e3-b2fb-40fd1a687021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:18.235227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6199ec-4dbf-46e3-b2fb-40fd1a687021 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:18.235685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6199ec-4dbf-46e3-b2fb-40fd1a687021 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:18.236092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6199ec-4dbf-46e3-b2fb-40fd1a687021 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:18.309975 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128a824-1ae0-41b6-be36-d7e41efe7d81 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:18.312055 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6199ec-4dbf-46e3-b2fb-40fd1a687021 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:18.313537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6199ec-4dbf-46e3-b2fb-40fd1a687021 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:18.316957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:18.341705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:18.348445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02172fb0-67b1-4246-b174-04480992019b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:18.349052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02172fb0-67b1-4246-b174-04480992019b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:18.349336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02172fb0-67b1-4246-b174-04480992019b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:18.349607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02172fb0-67b1-4246-b174-04480992019b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:18.350309 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02172fb0-67b1-4246-b174-04480992019b None admin] RBAC: Adding query filter params (interface=admin, service_id=edcbfb3293d442dcb5b537aa4a449951, region_id=RegionOne) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:18.350667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02172fb0-67b1-4246-b174-04480992019b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:18.352023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02172fb0-67b1-4246-b174-04480992019b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:18.357050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 504/982] 10.222.0.17 () {56 vars in 1249 bytes} [Fri Sep 30 19:17:18 2022] GET /identity/v3/endpoints?service_id=edcbfb3293d442dcb5b537aa4a449951&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:18.362634 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02172fb0-67b1-4246-b174-04480992019b None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:18.435487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 479/983] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:18 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:17:18.441167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6199ec-4dbf-46e3-b2fb-40fd1a687021 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:18.470907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02172fb0-67b1-4246-b174-04480992019b None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:18.507780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328fc44-6c4f-4601-b5d3-c78390d0fd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:18.508428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328fc44-6c4f-4601-b5d3-c78390d0fd36 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:18.508691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328fc44-6c4f-4601-b5d3-c78390d0fd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:18.508960 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328fc44-6c4f-4601-b5d3-c78390d0fd36 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:18.509599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328fc44-6c4f-4601-b5d3-c78390d0fd36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:18.511202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328fc44-6c4f-4601-b5d3-c78390d0fd36 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:18.517280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 505/984] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 30 19:17:18 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:17:18.543469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6199ec-4dbf-46e3-b2fb-40fd1a687021 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:18.547150 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:18.583015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f0c12-a943-464b-84c9-a16c99e91dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:18.583565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f0c12-a943-464b-84c9-a16c99e91dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:18.583899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f0c12-a943-464b-84c9-a16c99e91dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:18.584166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f0c12-a943-464b-84c9-a16c99e91dc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:18.597423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f0c12-a943-464b-84c9-a16c99e91dc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:18.598923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f0c12-a943-464b-84c9-a16c99e91dc2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7a7f0c12-a943-464b-84c9-a16c99e91dc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:18.615732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:18.616842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:18.617887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:18.617887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:18.617887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:18.620748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 480/985] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:18.622593 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328fc44-6c4f-4601-b5d3-c78390d0fd36 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:18.684025 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:18.729695 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328fc44-6c4f-4601-b5d3-c78390d0fd36 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:18.764443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ec620-4378-4705-8c37-5d2bbf7911bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:18.765243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ec620-4378-4705-8c37-5d2bbf7911bb None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:18.765660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ec620-4378-4705-8c37-5d2bbf7911bb None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:18.766101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ec620-4378-4705-8c37-5d2bbf7911bb None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:18.766874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ec620-4378-4705-8c37-5d2bbf7911bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:18.767331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ec620-4378-4705-8c37-5d2bbf7911bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:18.768838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ec620-4378-4705-8c37-5d2bbf7911bb None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:18.780555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 506/986] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:18.785883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f0c12-a943-464b-84c9-a16c99e91dc2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:18.886984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f0c12-a943-464b-84c9-a16c99e91dc2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:18.920173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17420f0-001f-4d15-b513-129d2aad1c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:18.920834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17420f0-001f-4d15-b513-129d2aad1c95 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:18.921110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17420f0-001f-4d15-b513-129d2aad1c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:18.921375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17420f0-001f-4d15-b513-129d2aad1c95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:18.928075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17420f0-001f-4d15-b513-129d2aad1c95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:18.930071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17420f0-001f-4d15-b513-129d2aad1c95 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e17420f0-001f-4d15-b513-129d2aad1c95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:17:18.941164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:18.942058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:18.943094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:17:18.943094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:17:18.943094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:17:18.943094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:17:18.943094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:17:18.943094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:17:18.943094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:18.943094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:18.943094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 481/987] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:18.948490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ec620-4378-4705-8c37-5d2bbf7911bb None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:19.047736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ec620-4378-4705-8c37-5d2bbf7911bb None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:19.065363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:19.083593 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4427a3-a83d-48d1-89f8-71a612313cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:19.084325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4427a3-a83d-48d1-89f8-71a612313cef None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:19.084705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4427a3-a83d-48d1-89f8-71a612313cef None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:19.085075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4427a3-a83d-48d1-89f8-71a612313cef None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:19.085880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4427a3-a83d-48d1-89f8-71a612313cef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:19.086242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4427a3-a83d-48d1-89f8-71a612313cef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:19.087713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4427a3-a83d-48d1-89f8-71a612313cef None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:19.099656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 507/988] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:19.105906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17420f0-001f-4d15-b513-129d2aad1c95 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:19.125057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:19.207728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17420f0-001f-4d15-b513-129d2aad1c95 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:19.242312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a643dc-f6a0-480c-a00c-3cf12db24535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:19.242910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a643dc-f6a0-480c-a00c-3cf12db24535 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:19.243202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a643dc-f6a0-480c-a00c-3cf12db24535 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:19.243456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a643dc-f6a0-480c-a00c-3cf12db24535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:19.244177 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a643dc-f6a0-480c-a00c-3cf12db24535 None admin] RBAC: Adding query filter params (role.id=657af95d19214896a42b23c30ab7a220, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=b638b8149ae149e3aad92ed51be92588) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:19.244523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a643dc-f6a0-480c-a00c-3cf12db24535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:19.245946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a643dc-f6a0-480c-a00c-3cf12db24535 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:19.260394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 482/989] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:17:19 2022] GET /identity/v3/role_assignments?user.id=b638b8149ae149e3aad92ed51be92588&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=657af95d19214896a42b23c30ab7a220 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:19.892043 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f0f99-42cc-4302-a00f-bde8283b5d15 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:19.892621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f0f99-42cc-4302-a00f-bde8283b5d15 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:19.892621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f0f99-42cc-4302-a00f-bde8283b5d15 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:19.894080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 508/990] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:19.899186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a1131-5e1e-4b33-aa4b-6a2be6de8c3e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:19.899470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a1131-5e1e-4b33-aa4b-6a2be6de8c3e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:19.899739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a1131-5e1e-4b33-aa4b-6a2be6de8c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:19.918296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-089a1131-5e1e-4b33-aa4b-6a2be6de8c3e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:19.919456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-089a1131-5e1e-4b33-aa4b-6a2be6de8c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:19.986299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 483/991] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:20.256451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f0f99-42cc-4302-a00f-bde8283b5d15 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:20.362221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f0f99-42cc-4302-a00f-bde8283b5d15 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:20.405829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8aae10-438a-48be-9fd2-7b1727f91bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:20.406468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aae10-438a-48be-9fd2-7b1727f91bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:20.406746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aae10-438a-48be-9fd2-7b1727f91bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:20.407009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aae10-438a-48be-9fd2-7b1727f91bc5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:20.407580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8aae10-438a-48be-9fd2-7b1727f91bc5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:20.408951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8aae10-438a-48be-9fd2-7b1727f91bc5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-3f8aae10-438a-48be-9fd2-7b1727f91bc5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:20.418620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:20.419626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:20.420570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 509/992] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:20.426166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a1131-5e1e-4b33-aa4b-6a2be6de8c3e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:20.438274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:20.531542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a1131-5e1e-4b33-aa4b-6a2be6de8c3e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:20.567648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118443d4-8067-460c-971c-b08026ab9110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:20.568274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118443d4-8067-460c-971c-b08026ab9110 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:20.568573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118443d4-8067-460c-971c-b08026ab9110 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:20.568838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118443d4-8067-460c-971c-b08026ab9110 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:20.569560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118443d4-8067-460c-971c-b08026ab9110 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:20.569859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118443d4-8067-460c-971c-b08026ab9110 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:20.571282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118443d4-8067-460c-971c-b08026ab9110 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:20.581462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 484/993] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:17:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:20.582792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8aae10-438a-48be-9fd2-7b1727f91bc5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:20.625983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:20.676467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdf729-1d90-49ce-9ef2-207aa6be553c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:20.676780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdf729-1d90-49ce-9ef2-207aa6be553c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:20.677022 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdf729-1d90-49ce-9ef2-207aa6be553c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:20.678346 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 485/994] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:20.683469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cac4a7-cd02-4e33-b4ac-2caf9d8d041d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:20.683701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8aae10-438a-48be-9fd2-7b1727f91bc5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:20.683847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cac4a7-cd02-4e33-b4ac-2caf9d8d041d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:20.684009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cac4a7-cd02-4e33-b4ac-2caf9d8d041d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:20.702529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-60cac4a7-cd02-4e33-b4ac-2caf9d8d041d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:20.703678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-60cac4a7-cd02-4e33-b4ac-2caf9d8d041d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:20.719673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba148e-f243-4056-b686-a98cc0341c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:20.720278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba148e-f243-4056-b686-a98cc0341c53 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:20.720736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba148e-f243-4056-b686-a98cc0341c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:20.721011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba148e-f243-4056-b686-a98cc0341c53 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:20.721927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba148e-f243-4056-b686-a98cc0341c53 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:20.722230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba148e-f243-4056-b686-a98cc0341c53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:20.723573 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba148e-f243-4056-b686-a98cc0341c53 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:20.730352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 510/995] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:17:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:20.736066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba148e-f243-4056-b686-a98cc0341c53 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:20.782938 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 486/996] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:20.838151 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba148e-f243-4056-b686-a98cc0341c53 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:20.845477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:20.872470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f823c11-9f3d-4976-873e-b6469f293fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:20.873071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f823c11-9f3d-4976-873e-b6469f293fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:20.873347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f823c11-9f3d-4976-873e-b6469f293fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:20.873614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f823c11-9f3d-4976-873e-b6469f293fa9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:20.874301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f823c11-9f3d-4976-873e-b6469f293fa9 None admin] RBAC: Adding query filter params (interface=public, service_id=ead86424910b4655803df6871ef757ed, region_id=RegionOne) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:20.874604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f823c11-9f3d-4976-873e-b6469f293fa9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:20.875956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f823c11-9f3d-4976-873e-b6469f293fa9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:20.881106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 511/997] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 30 19:17:20 2022] GET /identity/v3/endpoints?service_id=ead86424910b4655803df6871ef757ed&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:20.887224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cac4a7-cd02-4e33-b4ac-2caf9d8d041d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:20.891595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:20.988579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cac4a7-cd02-4e33-b4ac-2caf9d8d041d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:21.022180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e010b-00e4-46f1-a179-ba0c592c22e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:21.023298 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e010b-00e4-46f1-a179-ba0c592c22e7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:21.023575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e010b-00e4-46f1-a179-ba0c592c22e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:21.023846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e010b-00e4-46f1-a179-ba0c592c22e7 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:21.024466 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e010b-00e4-46f1-a179-ba0c592c22e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:21.025823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e010b-00e4-46f1-a179-ba0c592c22e7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:21.040711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 487/998] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 30 19:17:20 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:17:21.050195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f823c11-9f3d-4976-873e-b6469f293fa9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:21.160745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f823c11-9f3d-4976-873e-b6469f293fa9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:21.196784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904253b-7c22-4d62-9c50-a83d613d0079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:21.197418 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904253b-7c22-4d62-9c50-a83d613d0079 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:21.197767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904253b-7c22-4d62-9c50-a83d613d0079 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:21.198041 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904253b-7c22-4d62-9c50-a83d613d0079 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:21.198622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904253b-7c22-4d62-9c50-a83d613d0079 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:21.199977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904253b-7c22-4d62-9c50-a83d613d0079 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-2904253b-7c22-4d62-9c50-a83d613d0079 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:21.209066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:21.210601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:21.212651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 512/999] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Sep 30 19:17:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:21.216980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e010b-00e4-46f1-a179-ba0c592c22e7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:21.284764 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:21.322596 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e010b-00e4-46f1-a179-ba0c592c22e7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:21.358313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2293c-54c7-4acf-b020-655fab14db16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:21.358966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2293c-54c7-4acf-b020-655fab14db16 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:21.359244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2293c-54c7-4acf-b020-655fab14db16 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:21.359512 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2293c-54c7-4acf-b020-655fab14db16 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:21.360126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2293c-54c7-4acf-b020-655fab14db16 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:21.360450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2293c-54c7-4acf-b020-655fab14db16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:21.361795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2293c-54c7-4acf-b020-655fab14db16 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:21.367475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 488/1000] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 30 19:17:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:21.373090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904253b-7c22-4d62-9c50-a83d613d0079 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:21.414566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:21.433347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db561653-dd84-4db6-bbd4-861cb8aadc12 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:21.433899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db561653-dd84-4db6-bbd4-861cb8aadc12 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:21.433981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db561653-dd84-4db6-bbd4-861cb8aadc12 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:21.435310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 489/1001] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:21.440427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88892e64-0fe6-4cda-bf0b-c919e2de6344 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:21.440709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88892e64-0fe6-4cda-bf0b-c919e2de6344 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:21.440982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88892e64-0fe6-4cda-bf0b-c919e2de6344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:21.459068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-88892e64-0fe6-4cda-bf0b-c919e2de6344 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:21.460842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-88892e64-0fe6-4cda-bf0b-c919e2de6344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:21.487993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904253b-7c22-4d62-9c50-a83d613d0079 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:21.525760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30d7ca-a757-4fee-b0b9-819c6628bb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:21.526004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30d7ca-a757-4fee-b0b9-819c6628bb3d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:21.526305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30d7ca-a757-4fee-b0b9-819c6628bb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:21.526578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30d7ca-a757-4fee-b0b9-819c6628bb3d None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:21.527965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30d7ca-a757-4fee-b0b9-819c6628bb3d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:21.527965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30d7ca-a757-4fee-b0b9-819c6628bb3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:21.528935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30d7ca-a757-4fee-b0b9-819c6628bb3d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:21.532841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 490/1002] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:21.534445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 513/1003] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Sep 30 19:17:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:21.540141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88892e64-0fe6-4cda-bf0b-c919e2de6344 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:21.601980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:21.644732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88892e64-0fe6-4cda-bf0b-c919e2de6344 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:21.698649 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8679d-af3c-40c7-871c-1e2fc2c6026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:21.699619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8679d-af3c-40c7-871c-1e2fc2c6026c None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:21.700071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8679d-af3c-40c7-871c-1e2fc2c6026c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:21.700562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8679d-af3c-40c7-871c-1e2fc2c6026c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:21.701707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8679d-af3c-40c7-871c-1e2fc2c6026c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:21.704089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8679d-af3c-40c7-871c-1e2fc2c6026c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:21.724854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 491/1004] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:17:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:17:21.934257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30d7ca-a757-4fee-b0b9-819c6628bb3d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:22.034498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:22.038521 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30d7ca-a757-4fee-b0b9-819c6628bb3d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:22.079759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ff473-f541-43af-a3ba-fd67044c21f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:22.080526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ff473-f541-43af-a3ba-fd67044c21f1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:22.080689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ff473-f541-43af-a3ba-fd67044c21f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:22.080933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ff473-f541-43af-a3ba-fd67044c21f1 None admin] PATH_INFO: `/v3/services/image` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:22.081498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ff473-f541-43af-a3ba-fd67044c21f1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:22.082828 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ff473-f541-43af-a3ba-fd67044c21f1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-400ff473-f541-43af-a3ba-fd67044c21f1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:22.091423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:22.092495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:22.093381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 514/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Sep 30 19:17:21 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:22.100075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8679d-af3c-40c7-871c-1e2fc2c6026c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:22.213586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8679d-af3c-40c7-871c-1e2fc2c6026c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:22.236626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:22.251322 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf49a3-2b45-4bc9-a3b0-a76382f59630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:22.251932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf49a3-2b45-4bc9-a3b0-a76382f59630 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:22.252210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf49a3-2b45-4bc9-a3b0-a76382f59630 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:22.252513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf49a3-2b45-4bc9-a3b0-a76382f59630 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:22.253142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf49a3-2b45-4bc9-a3b0-a76382f59630 None admin] RBAC: Adding query filter params (name=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:22.253443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf49a3-2b45-4bc9-a3b0-a76382f59630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:22.254798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf49a3-2b45-4bc9-a3b0-a76382f59630 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:22.264156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 492/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:22.265340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ff473-f541-43af-a3ba-fd67044c21f1 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:22.370645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ff473-f541-43af-a3ba-fd67044c21f1 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:22.409887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655d37a-37a4-4eeb-a805-15dcf5be4955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:22.410490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655d37a-37a4-4eeb-a805-15dcf5be4955 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:22.410761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655d37a-37a4-4eeb-a805-15dcf5be4955 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:22.411029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655d37a-37a4-4eeb-a805-15dcf5be4955 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:22.411643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655d37a-37a4-4eeb-a805-15dcf5be4955 None admin] RBAC: Adding query filter params (type=image) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:22.411937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655d37a-37a4-4eeb-a805-15dcf5be4955 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:22.414070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655d37a-37a4-4eeb-a805-15dcf5be4955 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:22.418854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 515/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:22 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:23.223774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac41cd6-be80-47be-870e-82c4c2626fce None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:23.224318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac41cd6-be80-47be-870e-82c4c2626fce None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:23.224318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac41cd6-be80-47be-870e-82c4c2626fce None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:23.225741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 493/1008] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:23.230956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf08868-aa9e-4250-8e79-adc96f2a5b49 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:23.231348 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf08868-aa9e-4250-8e79-adc96f2a5b49 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:23.231525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf08868-aa9e-4250-8e79-adc96f2a5b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:23.249674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8cf08868-aa9e-4250-8e79-adc96f2a5b49 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:23.250838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8cf08868-aa9e-4250-8e79-adc96f2a5b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:23.267127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:23.320238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 516/1009] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:23.590251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac41cd6-be80-47be-870e-82c4c2626fce None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:23.694502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac41cd6-be80-47be-870e-82c4c2626fce None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:23.728416 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6a574-db6e-41f4-aded-53ca88299aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:23.729039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6a574-db6e-41f4-aded-53ca88299aab None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:23.729332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6a574-db6e-41f4-aded-53ca88299aab None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:23.729636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6a574-db6e-41f4-aded-53ca88299aab None admin] PATH_INFO: `/v3/services/compute` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:23.730217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6a574-db6e-41f4-aded-53ca88299aab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:23.731627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6a574-db6e-41f4-aded-53ca88299aab None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-9be6a574-db6e-41f4-aded-53ca88299aab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:23.740333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:23.741964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 19:17:23.742872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:23.744698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 494/1010] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:23.748053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf08868-aa9e-4250-8e79-adc96f2a5b49 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:23.843842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf08868-aa9e-4250-8e79-adc96f2a5b49 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:23.880727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d887f-5f58-488d-83fd-bd0f1ee9c67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:23.881362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d887f-5f58-488d-83fd-bd0f1ee9c67d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:23.881638 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d887f-5f58-488d-83fd-bd0f1ee9c67d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:23.881951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d887f-5f58-488d-83fd-bd0f1ee9c67d None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:23.882590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d887f-5f58-488d-83fd-bd0f1ee9c67d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:23.882888 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d887f-5f58-488d-83fd-bd0f1ee9c67d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:23.884262 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d887f-5f58-488d-83fd-bd0f1ee9c67d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:23.889714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 517/1011] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:17:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:23.895344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6a574-db6e-41f4-aded-53ca88299aab None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:23.945760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:23.975873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f6a9a-9705-4cc7-9a9d-7395d369fb9e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:23.976193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f6a9a-9705-4cc7-9a9d-7395d369fb9e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:23.976510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f6a9a-9705-4cc7-9a9d-7395d369fb9e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:23.978005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 518/1012] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:23.982864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba96ced-bac6-4d90-91ca-4137c912b77f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:23.983141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba96ced-bac6-4d90-91ca-4137c912b77f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:23.983417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba96ced-bac6-4d90-91ca-4137c912b77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:24.003755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-aba96ced-bac6-4d90-91ca-4137c912b77f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:24.004949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6a574-db6e-41f4-aded-53ca88299aab None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:24.005080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-aba96ced-bac6-4d90-91ca-4137c912b77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:24.020048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:24.040872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d07386-1503-4743-a5a2-761c78f2308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:24.041482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d07386-1503-4743-a5a2-761c78f2308b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:24.041764 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d07386-1503-4743-a5a2-761c78f2308b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:24.042036 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d07386-1503-4743-a5a2-761c78f2308b None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:24.042666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d07386-1503-4743-a5a2-761c78f2308b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:24.042966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d07386-1503-4743-a5a2-761c78f2308b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:24.044332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d07386-1503-4743-a5a2-761c78f2308b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:24.049481 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 495/1013] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Sep 30 19:17:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:24.056849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d07386-1503-4743-a5a2-761c78f2308b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:24.091054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 519/1014] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:24.163990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d07386-1503-4743-a5a2-761c78f2308b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:24.178129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:24.198499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7545510-b362-4afc-8564-2c6f67b84b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:24.199097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7545510-b362-4afc-8564-2c6f67b84b36 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:24.199370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7545510-b362-4afc-8564-2c6f67b84b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:24.199632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7545510-b362-4afc-8564-2c6f67b84b36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:24.200284 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7545510-b362-4afc-8564-2c6f67b84b36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:24.201672 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7545510-b362-4afc-8564-2c6f67b84b36 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:24.228391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 496/1015] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:17:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:17:24.359907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba96ced-bac6-4d90-91ca-4137c912b77f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:24.469674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba96ced-bac6-4d90-91ca-4137c912b77f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:24.514959 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce1adc-a135-468a-a2fe-e831d80422e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:24.515601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1adc-a135-468a-a2fe-e831d80422e6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:24.515879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1adc-a135-468a-a2fe-e831d80422e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:24.516158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1adc-a135-468a-a2fe-e831d80422e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:24.517302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce1adc-a135-468a-a2fe-e831d80422e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:24.518724 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce1adc-a135-468a-a2fe-e831d80422e6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:24.520471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 520/1016] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:24.526753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7545510-b362-4afc-8564-2c6f67b84b36 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:24.581116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7712947-8e3d-4232-92f6-1484ec473cc3 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:24.581442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7712947-8e3d-4232-92f6-1484ec473cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:24.581674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7712947-8e3d-4232-92f6-1484ec473cc3 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:24.583672 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 521/1017] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:24.589494 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e893ca-2de5-4823-9a4d-123a05071d8c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:24.590063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e893ca-2de5-4823-9a4d-123a05071d8c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:24.590336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e893ca-2de5-4823-9a4d-123a05071d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:24.613273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-28e893ca-2de5-4823-9a4d-123a05071d8c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:24.615156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-28e893ca-2de5-4823-9a4d-123a05071d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:24.618834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:24.633755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7545510-b362-4afc-8564-2c6f67b84b36 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:24.668825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9d138-55a8-48ca-89ab-ec33af5efa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:24.669425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9d138-55a8-48ca-89ab-ec33af5efa0d None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:24.669698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9d138-55a8-48ca-89ab-ec33af5efa0d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:24.669956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9d138-55a8-48ca-89ab-ec33af5efa0d None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:24.670591 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9d138-55a8-48ca-89ab-ec33af5efa0d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:24.672010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9d138-55a8-48ca-89ab-ec33af5efa0d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:24.689189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 522/1018] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:24 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:24.699591 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 497/1019] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:17:24 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:17:24.957153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e893ca-2de5-4823-9a4d-123a05071d8c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:25.055826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e893ca-2de5-4823-9a4d-123a05071d8c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:25.090482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cabeb-e7fb-4cb4-9229-14a1710e9d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:25.091093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cabeb-e7fb-4cb4-9229-14a1710e9d93 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:25.091371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cabeb-e7fb-4cb4-9229-14a1710e9d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:25.091641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cabeb-e7fb-4cb4-9229-14a1710e9d93 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:25.092295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841cabeb-e7fb-4cb4-9229-14a1710e9d93 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:25.093707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841cabeb-e7fb-4cb4-9229-14a1710e9d93 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:25.106732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:25.113377 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 523/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:17:24 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:17:26.460858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f4af3-e4ec-498a-a306-7f79d5091488 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:26.461400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f4af3-e4ec-498a-a306-7f79d5091488 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:26.461400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f4af3-e4ec-498a-a306-7f79d5091488 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:26.462877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 498/1021] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:26.467808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3667c0-cde3-4d4e-a4e6-5475f99d6853 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:26.468225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3667c0-cde3-4d4e-a4e6-5475f99d6853 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:26.468553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3667c0-cde3-4d4e-a4e6-5475f99d6853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:26.486864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fc3667c0-cde3-4d4e-a4e6-5475f99d6853 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:26.488017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fc3667c0-cde3-4d4e-a4e6-5475f99d6853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:26.560645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 524/1022] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:26.822454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f4af3-e4ec-498a-a306-7f79d5091488 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:26.846471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:26.911707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ccb00-5a50-4a16-9265-c44963ca644d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:26.912026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ccb00-5a50-4a16-9265-c44963ca644d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:26.912242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ccb00-5a50-4a16-9265-c44963ca644d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:26.914646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 525/1023] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:26.919541 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5567180-5f6d-4e1b-a699-b07f4523be39 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:26.919824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5567180-5f6d-4e1b-a699-b07f4523be39 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:26.920087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5567180-5f6d-4e1b-a699-b07f4523be39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:26.926780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f4af3-e4ec-498a-a306-7f79d5091488 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:26.937921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a5567180-5f6d-4e1b-a699-b07f4523be39 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:26.939076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a5567180-5f6d-4e1b-a699-b07f4523be39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:26.963316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5347ad2-c59f-4a3b-9e0c-8752c08efcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:26.963973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347ad2-c59f-4a3b-9e0c-8752c08efcd4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:26.964219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347ad2-c59f-4a3b-9e0c-8752c08efcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:26.964517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5347ad2-c59f-4a3b-9e0c-8752c08efcd4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:26.974605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5347ad2-c59f-4a3b-9e0c-8752c08efcd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d5347ad2-c59f-4a3b-9e0c-8752c08efcd4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:17:26.979801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:26.981347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:26.982600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 499/1024] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 30 19:17:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:26.986656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5347ad2-c59f-4a3b-9e0c-8752c08efcd4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:27.011804 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 526/1025] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:27.067827 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:27.087226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5347ad2-c59f-4a3b-9e0c-8752c08efcd4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:27.120517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd7b1c-5be5-48bc-884b-195877605128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:27.121117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd7b1c-5be5-48bc-884b-195877605128 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:27.121391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd7b1c-5be5-48bc-884b-195877605128 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:27.121659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd7b1c-5be5-48bc-884b-195877605128 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:27.122312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd7b1c-5be5-48bc-884b-195877605128 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:27.122608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd7b1c-5be5-48bc-884b-195877605128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:27.123984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd7b1c-5be5-48bc-884b-195877605128 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:27.136284 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 500/1026] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 30 19:17:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:27.137991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5567180-5f6d-4e1b-a699-b07f4523be39 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:27.198559 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:27.242091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5567180-5f6d-4e1b-a699-b07f4523be39 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:27.275241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5e024-09c9-4e6e-945f-3a56fd74799c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:27.275880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5e024-09c9-4e6e-945f-3a56fd74799c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:27.276156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5e024-09c9-4e6e-945f-3a56fd74799c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:27.276453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5e024-09c9-4e6e-945f-3a56fd74799c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:27.282208 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eedfa4-0ef1-47e6-abc0-089b9c67aed7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:27.282498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eedfa4-0ef1-47e6-abc0-089b9c67aed7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:27.282773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eedfa4-0ef1-47e6-abc0-089b9c67aed7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:27.284183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 501/1027] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:27.285010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eedfa4-0ef1-47e6-abc0-089b9c67aed7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:27.290408 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5e024-09c9-4e6e-945f-3a56fd74799c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:27.291844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5e024-09c9-4e6e-945f-3a56fd74799c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-57e5e024-09c9-4e6e-945f-3a56fd74799c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:27.307690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:27.308871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:27.310003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:27.310003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:27.310003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:27.310003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 527/1028] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:27.310003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f329d-a03c-4a02-a244-3fec24a2dabf None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:27.310435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f329d-a03c-4a02-a244-3fec24a2dabf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:27.310525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f329d-a03c-4a02-a244-3fec24a2dabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:27.327993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5e6f329d-a03c-4a02-a244-3fec24a2dabf None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:27.329189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5e6f329d-a03c-4a02-a244-3fec24a2dabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:27.393499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eedfa4-0ef1-47e6-abc0-089b9c67aed7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:27.393880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 528/1029] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:27.394694 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f329d-a03c-4a02-a244-3fec24a2dabf None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:27.432668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703805d-3f3b-4445-8e21-6627da37bdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:27.433868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703805d-3f3b-4445-8e21-6627da37bdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:27.433868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703805d-3f3b-4445-8e21-6627da37bdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:27.433868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703805d-3f3b-4445-8e21-6627da37bdf1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:27.434902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703805d-3f3b-4445-8e21-6627da37bdf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:27.436326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703805d-3f3b-4445-8e21-6627da37bdf1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:27.440798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 502/1030] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:27.440937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:27.444263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703805d-3f3b-4445-8e21-6627da37bdf1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:27.463025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:27.509102 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f329d-a03c-4a02-a244-3fec24a2dabf None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:27.545985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6e3f-ef76-4cb7-9def-7d4ed8ad0444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:27.546863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6e3f-ef76-4cb7-9def-7d4ed8ad0444 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:27.547135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6e3f-ef76-4cb7-9def-7d4ed8ad0444 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:27.547403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd6e3f-ef76-4cb7-9def-7d4ed8ad0444 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:27.548033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd6e3f-ef76-4cb7-9def-7d4ed8ad0444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:27.548331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd6e3f-ef76-4cb7-9def-7d4ed8ad0444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:27.549748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd6e3f-ef76-4cb7-9def-7d4ed8ad0444 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:27.558502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703805d-3f3b-4445-8e21-6627da37bdf1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:27.561386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 529/1031] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:27.566510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6e3f-ef76-4cb7-9def-7d4ed8ad0444 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:27.594231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb4bd3-3423-4ca1-b03c-cb05d09f8bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:27.594836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb4bd3-3423-4ca1-b03c-cb05d09f8bcf None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:27.595116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb4bd3-3423-4ca1-b03c-cb05d09f8bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:27.595389 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb4bd3-3423-4ca1-b03c-cb05d09f8bcf None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:27.596028 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb4bd3-3423-4ca1-b03c-cb05d09f8bcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:27.597497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb4bd3-3423-4ca1-b03c-cb05d09f8bcf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:27.620457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 503/1032] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:17:27 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:17:27.660774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb4bd3-3423-4ca1-b03c-cb05d09f8bcf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:27.678403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:27.684450 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd6e3f-ef76-4cb7-9def-7d4ed8ad0444 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:27.724017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d93cf-dff8-402f-8d95-00f1260b4101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:27.724700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d93cf-dff8-402f-8d95-00f1260b4101 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:27.724979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d93cf-dff8-402f-8d95-00f1260b4101 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:27.725246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d93cf-dff8-402f-8d95-00f1260b4101 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:27.733703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d93cf-dff8-402f-8d95-00f1260b4101 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:27.735232 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d93cf-dff8-402f-8d95-00f1260b4101 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:27.737271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-433d93cf-dff8-402f-8d95-00f1260b4101 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:17:27.749353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:27.750375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:27.751850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:27.751850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:27.751850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:17:27.751850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:17:27.751850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:17:27.751850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:17:27.751850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:17:27.751850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:17:27.751850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:17:27.751850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:27.751850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 530/1033] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 30 19:17:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:27.756748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d93cf-dff8-402f-8d95-00f1260b4101 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:27.785385 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb4bd3-3423-4ca1-b03c-cb05d09f8bcf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:27.839670 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e821fe-a8a9-404c-8754-1e53be619d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:27.840308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e821fe-a8a9-404c-8754-1e53be619d20 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:27.840616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e821fe-a8a9-404c-8754-1e53be619d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:27.840890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e821fe-a8a9-404c-8754-1e53be619d20 None admin] PATH_INFO: `/v3/services/image` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:27.841480 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e821fe-a8a9-404c-8754-1e53be619d20 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:27.842890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e821fe-a8a9-404c-8754-1e53be619d20 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e7e821fe-a8a9-404c-8754-1e53be619d20 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:27.852396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:27.853458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:27.854330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 504/1034] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Sep 30 19:17:27 2022] GET /identity/v3/services/image => generated 86 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:27.859620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e821fe-a8a9-404c-8754-1e53be619d20 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:27.886389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d93cf-dff8-402f-8d95-00f1260b4101 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:27.924831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15314879-abb6-4a82-aa7f-90a68869f7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:27.925475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15314879-abb6-4a82-aa7f-90a68869f7bc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:27.925872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15314879-abb6-4a82-aa7f-90a68869f7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:27.926138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15314879-abb6-4a82-aa7f-90a68869f7bc None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:27.926978 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15314879-abb6-4a82-aa7f-90a68869f7bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:27.927279 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15314879-abb6-4a82-aa7f-90a68869f7bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:27.928873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15314879-abb6-4a82-aa7f-90a68869f7bc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:27.951818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 531/1035] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 30 19:17:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:27.952871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15314879-abb6-4a82-aa7f-90a68869f7bc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:27.971777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e821fe-a8a9-404c-8754-1e53be619d20 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:27.971993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:28.002087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:28.008417 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938d9d0-e76d-4c52-a79c-ad05366be6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:28.009105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d9d0-e76d-4c52-a79c-ad05366be6ca None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:28.009450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d9d0-e76d-4c52-a79c-ad05366be6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:28.009783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d9d0-e76d-4c52-a79c-ad05366be6ca None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:28.010481 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938d9d0-e76d-4c52-a79c-ad05366be6ca None admin] RBAC: Adding query filter params (name=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:28.010847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938d9d0-e76d-4c52-a79c-ad05366be6ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:28.012293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938d9d0-e76d-4c52-a79c-ad05366be6ca None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:28.020708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 505/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:28.022878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938d9d0-e76d-4c52-a79c-ad05366be6ca None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:28.065391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15314879-abb6-4a82-aa7f-90a68869f7bc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:28.101690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe767d85-7203-4ff9-8d31-95a3d9050608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:28.102301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe767d85-7203-4ff9-8d31-95a3d9050608 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:28.102574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe767d85-7203-4ff9-8d31-95a3d9050608 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:28.102841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe767d85-7203-4ff9-8d31-95a3d9050608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:28.138435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938d9d0-e76d-4c52-a79c-ad05366be6ca None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:28.173237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07451fd-d872-491d-bb23-8550dcd777d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:28.173842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07451fd-d872-491d-bb23-8550dcd777d9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:28.174122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07451fd-d872-491d-bb23-8550dcd777d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:28.174393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07451fd-d872-491d-bb23-8550dcd777d9 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:28.175073 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07451fd-d872-491d-bb23-8550dcd777d9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:28.175321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07451fd-d872-491d-bb23-8550dcd777d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:28.176726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07451fd-d872-491d-bb23-8550dcd777d9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:28.179810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe767d85-7203-4ff9-8d31-95a3d9050608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:28.181334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe767d85-7203-4ff9-8d31-95a3d9050608 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:28.187112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 506/1037] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:28.187914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07451fd-d872-491d-bb23-8550dcd777d9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:28.199079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:28.258969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:28.283677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 532/1038] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:17:28.290018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe767d85-7203-4ff9-8d31-95a3d9050608 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:28.309116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07451fd-d872-491d-bb23-8550dcd777d9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:28.344921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184d516-a82c-4172-902d-07e46865cdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:28.345668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184d516-a82c-4172-902d-07e46865cdd3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:28.346040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184d516-a82c-4172-902d-07e46865cdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:28.346412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184d516-a82c-4172-902d-07e46865cdd3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:28.347231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184d516-a82c-4172-902d-07e46865cdd3 None admin] RBAC: Adding query filter params (interface=public, service_id=de989329de344cd792105ae7c926f1c6, region_id=RegionOne) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:28.347644 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184d516-a82c-4172-902d-07e46865cdd3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:28.349094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184d516-a82c-4172-902d-07e46865cdd3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:28.354682 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 507/1039] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 30 19:17:28 2022] GET /identity/v3/endpoints?service_id=de989329de344cd792105ae7c926f1c6&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:28.360023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184d516-a82c-4172-902d-07e46865cdd3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:28.401212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe767d85-7203-4ff9-8d31-95a3d9050608 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:28.420400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:28.441211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3111dfe-e442-4048-8376-6001ee7e2835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:28.441844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3111dfe-e442-4048-8376-6001ee7e2835 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:28.442117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3111dfe-e442-4048-8376-6001ee7e2835 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:28.442382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3111dfe-e442-4048-8376-6001ee7e2835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:28.453667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3111dfe-e442-4048-8376-6001ee7e2835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:28.455343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3111dfe-e442-4048-8376-6001ee7e2835 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:28.469653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184d516-a82c-4172-902d-07e46865cdd3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d3111dfe-e442-4048-8376-6001ee7e2835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:28.471436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:28.472446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:28.473435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:28.473435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:28.473435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:28.473435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 533/1040] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:28.478383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3111dfe-e442-4048-8376-6001ee7e2835 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:28.496323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:28.509472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647ebb2-b5a9-437e-933b-75cd95e7fbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:28.510073 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647ebb2-b5a9-437e-933b-75cd95e7fbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:28.510354 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647ebb2-b5a9-437e-933b-75cd95e7fbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:28.510623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647ebb2-b5a9-437e-933b-75cd95e7fbf8 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:28.511223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647ebb2-b5a9-437e-933b-75cd95e7fbf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:28.512609 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647ebb2-b5a9-437e-933b-75cd95e7fbf8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:28.520326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 508/1041] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 30 19:17:28 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:17:28.588220 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3111dfe-e442-4048-8376-6001ee7e2835 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:28.624496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec840969-afaf-42f7-84d7-cadf1b18233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:28.625010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec840969-afaf-42f7-84d7-cadf1b18233e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:28.625285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec840969-afaf-42f7-84d7-cadf1b18233e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:28.625554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec840969-afaf-42f7-84d7-cadf1b18233e None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:28.626197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec840969-afaf-42f7-84d7-cadf1b18233e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:28.626498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec840969-afaf-42f7-84d7-cadf1b18233e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:28.627868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec840969-afaf-42f7-84d7-cadf1b18233e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:28.645652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 534/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:28.647658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647ebb2-b5a9-437e-933b-75cd95e7fbf8 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:28.755627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647ebb2-b5a9-437e-933b-75cd95e7fbf8 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:28.790374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa06034-b72b-4120-b115-fd1e2652b5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:28.790998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa06034-b72b-4120-b115-fd1e2652b5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:28.791287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa06034-b72b-4120-b115-fd1e2652b5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:28.791547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa06034-b72b-4120-b115-fd1e2652b5c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:28.798259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa06034-b72b-4120-b115-fd1e2652b5c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:28.799951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa06034-b72b-4120-b115-fd1e2652b5c2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-3aa06034-b72b-4120-b115-fd1e2652b5c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:17:28.811333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:28.812476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:28.818253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:17:28.818253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:17:28.818253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:17:28.818253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:17:28.818253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:17:28.818253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:17:28.818253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:28.818253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:28.818253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 509/1043] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:28.820188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec840969-afaf-42f7-84d7-cadf1b18233e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:28.849176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:28.922027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec840969-afaf-42f7-84d7-cadf1b18233e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:28.955078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a683f-3453-480c-bcbe-2a207853b667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:28.955680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a683f-3453-480c-bcbe-2a207853b667 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:28.955951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a683f-3453-480c-bcbe-2a207853b667 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:28.956214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a683f-3453-480c-bcbe-2a207853b667 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:28.956927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a683f-3453-480c-bcbe-2a207853b667 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:28.957223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a683f-3453-480c-bcbe-2a207853b667 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:28.958627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a683f-3453-480c-bcbe-2a207853b667 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:28.975637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 535/1044] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:28.977167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa06034-b72b-4120-b115-fd1e2652b5c2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:29.028546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:29.081038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa06034-b72b-4120-b115-fd1e2652b5c2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:29.115067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad37db-cd12-4958-a073-c53ec73a4dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:29.115666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad37db-cd12-4958-a073-c53ec73a4dac None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:29.115967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad37db-cd12-4958-a073-c53ec73a4dac None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:29.116244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad37db-cd12-4958-a073-c53ec73a4dac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:29.116991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad37db-cd12-4958-a073-c53ec73a4dac None admin] RBAC: Adding query filter params (role.id=657af95d19214896a42b23c30ab7a220, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=5952db29e6fc4ddbb9aeb0522497312c) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:29.117298 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad37db-cd12-4958-a073-c53ec73a4dac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:29.118710 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad37db-cd12-4958-a073-c53ec73a4dac None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:29.124238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 510/1045] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:17:28 2022] GET /identity/v3/role_assignments?user.id=5952db29e6fc4ddbb9aeb0522497312c&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=657af95d19214896a42b23c30ab7a220 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:29.865853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd2653-ca9b-443d-bdc7-a620358af2e0 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:29.866403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd2653-ca9b-443d-bdc7-a620358af2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:29.866403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd2653-ca9b-443d-bdc7-a620358af2e0 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:29.867902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 536/1046] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:29.872895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862ffee-4a68-42d5-bde1-891a98019fbc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:29.873290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862ffee-4a68-42d5-bde1-891a98019fbc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:29.877486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862ffee-4a68-42d5-bde1-891a98019fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:29.895755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8862ffee-4a68-42d5-bde1-891a98019fbc None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:29.896934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8862ffee-4a68-42d5-bde1-891a98019fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:29.962704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 511/1047] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:30.238090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd2653-ca9b-443d-bdc7-a620358af2e0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:30.327489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:30.342445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd2653-ca9b-443d-bdc7-a620358af2e0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:30.375497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10292048-25d2-42af-9423-9592a67cfca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:30.376125 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10292048-25d2-42af-9423-9592a67cfca2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:30.376418 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10292048-25d2-42af-9423-9592a67cfca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:30.376695 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10292048-25d2-42af-9423-9592a67cfca2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:30.391979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10292048-25d2-42af-9423-9592a67cfca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-10292048-25d2-42af-9423-9592a67cfca2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:17:30.398382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:30.399584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:17:30.400897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:30.405820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 537/1048] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:17:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:30.405965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862ffee-4a68-42d5-bde1-891a98019fbc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:30.448161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:30.507071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862ffee-4a68-42d5-bde1-891a98019fbc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:30.541061 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7c616-8953-42e0-b95e-47ae91693924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:30.541667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7c616-8953-42e0-b95e-47ae91693924 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:30.541942 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7c616-8953-42e0-b95e-47ae91693924 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:30.542210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7c616-8953-42e0-b95e-47ae91693924 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:30.542863 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7c616-8953-42e0-b95e-47ae91693924 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:30.543163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7c616-8953-42e0-b95e-47ae91693924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:30.544542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7c616-8953-42e0-b95e-47ae91693924 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:30.556586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 512/1049] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:17:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:30.559421 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10292048-25d2-42af-9423-9592a67cfca2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:30.664665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10292048-25d2-42af-9423-9592a67cfca2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:30.697445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32541939-7405-4cd2-b2dc-41f2753bdc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:30.698071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32541939-7405-4cd2-b2dc-41f2753bdc68 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:30.698342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32541939-7405-4cd2-b2dc-41f2753bdc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:30.698608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32541939-7405-4cd2-b2dc-41f2753bdc68 None admin] PATH_INFO: `/v3/users/678ff473fc9d4f409a8887558437a341` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:30.703556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebe99b-101d-4bef-bbe8-0ff1b9b249c9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:30.703842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebe99b-101d-4bef-bbe8-0ff1b9b249c9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:30.704117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebe99b-101d-4bef-bbe8-0ff1b9b249c9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:30.705625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 513/1050] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:30.711249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4b69a-2cbf-4ca1-8686-502850642c41 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:30.711533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4b69a-2cbf-4ca1-8686-502850642c41 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:30.711807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4b69a-2cbf-4ca1-8686-502850642c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:30.713519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32541939-7405-4cd2-b2dc-41f2753bdc68 None admin] RBAC: Authorizing `identity:get_user(user_id=678ff473fc9d4f409a8887558437a341)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:30.714946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32541939-7405-4cd2-b2dc-41f2753bdc68 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:30.721430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 538/1051] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:17:30 2022] GET /identity/v3/users/678ff473fc9d4f409a8887558437a341 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:30.722074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32541939-7405-4cd2-b2dc-41f2753bdc68 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:30.731498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:30.733466 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b5b4b69a-2cbf-4ca1-8686-502850642c41 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:30.735046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b5b4b69a-2cbf-4ca1-8686-502850642c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:30.804953 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 514/1052] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:30.835160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32541939-7405-4cd2-b2dc-41f2753bdc68 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:30.869351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540895b-4ee1-481b-b94f-3351c723206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:30.869955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540895b-4ee1-481b-b94f-3351c723206d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:30.870214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540895b-4ee1-481b-b94f-3351c723206d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:30.870537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540895b-4ee1-481b-b94f-3351c723206d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:30.936540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540895b-4ee1-481b-b94f-3351c723206d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:30.937973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540895b-4ee1-481b-b94f-3351c723206d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:30.945409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:31.039475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 539/1053] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:30 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 30 19:17:31.040651 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4b69a-2cbf-4ca1-8686-502850642c41 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:31.073658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:31.075339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540895b-4ee1-481b-b94f-3351c723206d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:31.150468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4b69a-2cbf-4ca1-8686-502850642c41 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:31.183333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540895b-4ee1-481b-b94f-3351c723206d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:31.187366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c4b93-923b-48ad-9f23-895e512f7aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:31.188011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c4b93-923b-48ad-9f23-895e512f7aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:31.188276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c4b93-923b-48ad-9f23-895e512f7aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:31.188586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c4b93-923b-48ad-9f23-895e512f7aa4 None admin] PATH_INFO: `/v3/projects/8a13e7576fa646b5a7d29584e6e162c6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:31.189700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c4b93-923b-48ad-9f23-895e512f7aa4 None admin] RBAC: Authorizing `identity:get_project(project_id=8a13e7576fa646b5a7d29584e6e162c6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:31.190734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:31.191136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c4b93-923b-48ad-9f23-895e512f7aa4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:31.192819 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 515/1054] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:17:31 2022] GET /identity/v3/projects/8a13e7576fa646b5a7d29584e6e162c6 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:31.199374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c4b93-923b-48ad-9f23-895e512f7aa4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:31.226766 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdc7e2-e438-4c58-a9e0-3388335183f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:31.227404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdc7e2-e438-4c58-a9e0-3388335183f5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:31.227681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdc7e2-e438-4c58-a9e0-3388335183f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:31.227996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdc7e2-e438-4c58-a9e0-3388335183f5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:31.228561 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdc7e2-e438-4c58-a9e0-3388335183f5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:31.229914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdc7e2-e438-4c58-a9e0-3388335183f5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-8cfdc7e2-e438-4c58-a9e0-3388335183f5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:31.238764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:31.239784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 19:17:31.240720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:31.244539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 540/1055] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Sep 30 19:17:31 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:31.246330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdc7e2-e438-4c58-a9e0-3388335183f5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:31.303419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c4b93-923b-48ad-9f23-895e512f7aa4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:31.305527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:31.337763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79e551-7c4e-4838-b251-8a78aac870b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:31.338287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79e551-7c4e-4838-b251-8a78aac870b1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:31.338562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79e551-7c4e-4838-b251-8a78aac870b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:31.338828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79e551-7c4e-4838-b251-8a78aac870b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:31.339540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79e551-7c4e-4838-b251-8a78aac870b1 None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=8a13e7576fa646b5a7d29584e6e162c6, user.id=678ff473fc9d4f409a8887558437a341) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:31.339840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79e551-7c4e-4838-b251-8a78aac870b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:31.341279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79e551-7c4e-4838-b251-8a78aac870b1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:31.343106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdc7e2-e438-4c58-a9e0-3388335183f5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:31.346605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 516/1056] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:17:31 2022] GET /identity/v3/role_assignments?user.id=678ff473fc9d4f409a8887558437a341&scope.project.id=8a13e7576fa646b5a7d29584e6e162c6&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:31.347727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500e86b-d132-4c52-b34a-0a98ed61a999 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:31.348004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500e86b-d132-4c52-b34a-0a98ed61a999 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:31.348269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500e86b-d132-4c52-b34a-0a98ed61a999 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:31.349635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 517/1057] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:31.355728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705a9d-b8c2-4de9-be80-93e3840c82af None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:31.356024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705a9d-b8c2-4de9-be80-93e3840c82af None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:31.356283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705a9d-b8c2-4de9-be80-93e3840c82af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:31.380215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6d705a9d-b8c2-4de9-be80-93e3840c82af None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:31.381455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6d705a9d-b8c2-4de9-be80-93e3840c82af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:31.383178 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef2b83-3f2f-4502-b2b5-7696bcedc9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:31.383994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef2b83-3f2f-4502-b2b5-7696bcedc9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:31.384467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef2b83-3f2f-4502-b2b5-7696bcedc9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:31.384889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef2b83-3f2f-4502-b2b5-7696bcedc9a2 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:31.385683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef2b83-3f2f-4502-b2b5-7696bcedc9a2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:31.386138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef2b83-3f2f-4502-b2b5-7696bcedc9a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:31.387674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef2b83-3f2f-4502-b2b5-7696bcedc9a2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:31.393920 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 541/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:31.399429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef2b83-3f2f-4502-b2b5-7696bcedc9a2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:31.437772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:31.472560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 518/1059] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:31.510886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef2b83-3f2f-4502-b2b5-7696bcedc9a2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:31.546312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea04e9-f7c3-4586-bf94-6964b2a1cdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:31.546913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea04e9-f7c3-4586-bf94-6964b2a1cdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:31.547185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea04e9-f7c3-4586-bf94-6964b2a1cdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:31.547449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea04e9-f7c3-4586-bf94-6964b2a1cdd7 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:31.548058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea04e9-f7c3-4586-bf94-6964b2a1cdd7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:31.548350 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea04e9-f7c3-4586-bf94-6964b2a1cdd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:31.549708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea04e9-f7c3-4586-bf94-6964b2a1cdd7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:31.555079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 542/1060] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:31.560558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d705a9d-b8c2-4de9-be80-93e3840c82af None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:31.654402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:31.661539 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d705a9d-b8c2-4de9-be80-93e3840c82af None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:31.706099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13f4da-90e3-4879-9fc1-b71bea5c2346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:31.706711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13f4da-90e3-4879-9fc1-b71bea5c2346 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:31.706988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13f4da-90e3-4879-9fc1-b71bea5c2346 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:31.707256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13f4da-90e3-4879-9fc1-b71bea5c2346 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:31.707987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13f4da-90e3-4879-9fc1-b71bea5c2346 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:31.709464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13f4da-90e3-4879-9fc1-b71bea5c2346 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:31.736479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 519/1061] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:17:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:17:31.740174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea04e9-f7c3-4586-bf94-6964b2a1cdd7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:31.847437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:31.853417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea04e9-f7c3-4586-bf94-6964b2a1cdd7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:31.895390 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f84fe0-1de0-4898-84b6-3ce3ff3696dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:31.896191 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f84fe0-1de0-4898-84b6-3ce3ff3696dc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:31.896442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f84fe0-1de0-4898-84b6-3ce3ff3696dc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:31.896645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f84fe0-1de0-4898-84b6-3ce3ff3696dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:31.909556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f84fe0-1de0-4898-84b6-3ce3ff3696dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a7f84fe0-1de0-4898-84b6-3ce3ff3696dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:17:31.914040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:31.915009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:31.915911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 543/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 30 19:17:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:31.921315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13f4da-90e3-4879-9fc1-b71bea5c2346 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:32.023243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13f4da-90e3-4879-9fc1-b71bea5c2346 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:32.058431 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb1660-8af1-4c5b-93c7-7ca988c4d262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:32.059069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb1660-8af1-4c5b-93c7-7ca988c4d262 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:32.059327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb1660-8af1-4c5b-93c7-7ca988c4d262 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:32.059575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb1660-8af1-4c5b-93c7-7ca988c4d262 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:32.060281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb1660-8af1-4c5b-93c7-7ca988c4d262 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:32.060566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb1660-8af1-4c5b-93c7-7ca988c4d262 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:32.061964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb1660-8af1-4c5b-93c7-7ca988c4d262 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:32.070757 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 520/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 30 19:17:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:32.076634 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f84fe0-1de0-4898-84b6-3ce3ff3696dc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:32.178237 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f84fe0-1de0-4898-84b6-3ce3ff3696dc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:32.211863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20607f-4056-427c-88d7-fc5b163c4b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:32.212537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20607f-4056-427c-88d7-fc5b163c4b29 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:32.212817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20607f-4056-427c-88d7-fc5b163c4b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:32.213086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20607f-4056-427c-88d7-fc5b163c4b29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:32.224497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20607f-4056-427c-88d7-fc5b163c4b29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:32.226521 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20607f-4056-427c-88d7-fc5b163c4b29 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-aa20607f-4056-427c-88d7-fc5b163c4b29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:32.243468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:32.244600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:32.245540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:32.245540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:32.245540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:32.248695 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 544/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:32.250334 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb1660-8af1-4c5b-93c7-7ca988c4d262 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:32.267530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:32.356096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb1660-8af1-4c5b-93c7-7ca988c4d262 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:32.393448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1aa5d0-9722-4f80-8638-7efe28dd44eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:32.394054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1aa5d0-9722-4f80-8638-7efe28dd44eb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:32.394332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1aa5d0-9722-4f80-8638-7efe28dd44eb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:32.394601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1aa5d0-9722-4f80-8638-7efe28dd44eb None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:32.395235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1aa5d0-9722-4f80-8638-7efe28dd44eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:32.395534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1aa5d0-9722-4f80-8638-7efe28dd44eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:32.399993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1aa5d0-9722-4f80-8638-7efe28dd44eb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:32.412222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 521/1065] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:32.418782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20607f-4056-427c-88d7-fc5b163c4b29 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:32.428026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:32.525000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20607f-4056-427c-88d7-fc5b163c4b29 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:32.559911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d145e8-7a14-41ed-82a6-c7ac2cec2875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:32.560567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d145e8-7a14-41ed-82a6-c7ac2cec2875 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:32.560846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d145e8-7a14-41ed-82a6-c7ac2cec2875 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:32.561180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d145e8-7a14-41ed-82a6-c7ac2cec2875 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:32.568757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d145e8-7a14-41ed-82a6-c7ac2cec2875 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:32.570224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d145e8-7a14-41ed-82a6-c7ac2cec2875 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-06d145e8-7a14-41ed-82a6-c7ac2cec2875 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:17:32.581330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:32.582326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:32.583322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:32.583322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:32.583322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:17:32.583322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:17:32.583322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:17:32.583322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:17:32.583322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:17:32.583322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:17:32.583322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:17:32.583322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:32.587627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 545/1066] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 30 19:17:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:32.588816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1aa5d0-9722-4f80-8638-7efe28dd44eb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:32.671131 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:32.699626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1aa5d0-9722-4f80-8638-7efe28dd44eb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:32.734626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51289c39-acdd-429e-8c3a-f23999f387c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:32.735230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51289c39-acdd-429e-8c3a-f23999f387c2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:32.735505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51289c39-acdd-429e-8c3a-f23999f387c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:32.735771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51289c39-acdd-429e-8c3a-f23999f387c2 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:32.736514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51289c39-acdd-429e-8c3a-f23999f387c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:32.736813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51289c39-acdd-429e-8c3a-f23999f387c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:32.738219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51289c39-acdd-429e-8c3a-f23999f387c2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:32.760925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 522/1067] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 30 19:17:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:32.762345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d145e8-7a14-41ed-82a6-c7ac2cec2875 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:32.828583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:32.865474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d145e8-7a14-41ed-82a6-c7ac2cec2875 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:32.898157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5af6f-c3bf-496c-a751-a5dfcb7e37c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:32.898810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5af6f-c3bf-496c-a751-a5dfcb7e37c7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:32.899100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5af6f-c3bf-496c-a751-a5dfcb7e37c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:32.899368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5af6f-c3bf-496c-a751-a5dfcb7e37c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:32.909872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5af6f-c3bf-496c-a751-a5dfcb7e37c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:32.911311 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5af6f-c3bf-496c-a751-a5dfcb7e37c7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-59d5af6f-c3bf-496c-a751-a5dfcb7e37c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:32.925856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:32.926819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:32.927767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:32.927767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:32.927767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:32.927767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 546/1068] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:32.934478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51289c39-acdd-429e-8c3a-f23999f387c2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:33.035427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51289c39-acdd-429e-8c3a-f23999f387c2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:33.066577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:33.072874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a6e1b-3c59-479e-b486-99e61fb7b40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:33.073485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a6e1b-3c59-479e-b486-99e61fb7b40d None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:33.073762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a6e1b-3c59-479e-b486-99e61fb7b40d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:33.074034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a6e1b-3c59-479e-b486-99e61fb7b40d None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:33.074673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a6e1b-3c59-479e-b486-99e61fb7b40d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:33.075002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a6e1b-3c59-479e-b486-99e61fb7b40d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:33.076361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a6e1b-3c59-479e-b486-99e61fb7b40d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:33.092897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 523/1069] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:33.094012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5af6f-c3bf-496c-a751-a5dfcb7e37c7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:33.192848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5af6f-c3bf-496c-a751-a5dfcb7e37c7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:33.209075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:33.227245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572564a-6f18-402e-a015-1cc621a96f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:33.227871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572564a-6f18-402e-a015-1cc621a96f28 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:33.228148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572564a-6f18-402e-a015-1cc621a96f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:33.228440 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572564a-6f18-402e-a015-1cc621a96f28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:33.235017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572564a-6f18-402e-a015-1cc621a96f28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:33.236471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572564a-6f18-402e-a015-1cc621a96f28 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c572564a-6f18-402e-a015-1cc621a96f28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:17:33.247562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:33.253316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:33.254271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:17:33.254271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:17:33.254271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:17:33.254271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:17:33.254271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:17:33.254271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:17:33.254271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:33.254271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:33.254271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 547/1070] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:33.256262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a6e1b-3c59-479e-b486-99e61fb7b40d None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:33.358403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a6e1b-3c59-479e-b486-99e61fb7b40d None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:33.400221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cca97-3e2a-4124-aec4-b49e2ed34f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:33.400221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cca97-3e2a-4124-aec4-b49e2ed34f6b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:33.400221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cca97-3e2a-4124-aec4-b49e2ed34f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:33.400221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cca97-3e2a-4124-aec4-b49e2ed34f6b None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:33.400675 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cca97-3e2a-4124-aec4-b49e2ed34f6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:33.400675 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cca97-3e2a-4124-aec4-b49e2ed34f6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:33.401107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cca97-3e2a-4124-aec4-b49e2ed34f6b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:33.415666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 524/1071] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:33.421791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572564a-6f18-402e-a015-1cc621a96f28 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:33.514943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98cf10-b37a-4fd5-9e70-c78755652535 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:33.515223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98cf10-b37a-4fd5-9e70-c78755652535 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:33.515493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98cf10-b37a-4fd5-9e70-c78755652535 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:33.516874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 525/1072] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:33.521744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4f7c8-b7b9-4cbf-a158-6fbc1bd1245e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:33.522029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4f7c8-b7b9-4cbf-a158-6fbc1bd1245e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:33.522338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4f7c8-b7b9-4cbf-a158-6fbc1bd1245e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:33.524627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572564a-6f18-402e-a015-1cc621a96f28 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:33.540910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d2d4f7c8-b7b9-4cbf-a158-6fbc1bd1245e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:33.542076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d2d4f7c8-b7b9-4cbf-a158-6fbc1bd1245e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:33.564766 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab9bbd-50aa-4700-8a8c-57528ab03e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:33.565434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab9bbd-50aa-4700-8a8c-57528ab03e56 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:33.565714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab9bbd-50aa-4700-8a8c-57528ab03e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:33.565990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab9bbd-50aa-4700-8a8c-57528ab03e56 None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/5952db29e6fc4ddbb9aeb0522497312c/roles/657af95d19214896a42b23c30ab7a220` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:33.568147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab9bbd-50aa-4700-8a8c-57528ab03e56 None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=5952db29e6fc4ddbb9aeb0522497312c, role_id=657af95d19214896a42b23c30ab7a220)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:33.569663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab9bbd-50aa-4700-8a8c-57528ab03e56 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:33.583839 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 548/1073] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:17:33 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/5952db29e6fc4ddbb9aeb0522497312c/roles/657af95d19214896a42b23c30ab7a220 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:17:33.592415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:33.617110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 526/1074] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:33.883101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab9bbd-50aa-4700-8a8c-57528ab03e56 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:33.925843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:33.931136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee1b68-ebb7-4f86-b2e5-e24dcc47470f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:33.931426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee1b68-ebb7-4f86-b2e5-e24dcc47470f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:33.931703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee1b68-ebb7-4f86-b2e5-e24dcc47470f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:33.933559 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 527/1075] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:33.939096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f6650-0a7e-4fd7-87c6-f295e8c6253d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:33.939376 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f6650-0a7e-4fd7-87c6-f295e8c6253d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:33.939645 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f6650-0a7e-4fd7-87c6-f295e8c6253d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:33.958204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7f0f6650-0a7e-4fd7-87c6-f295e8c6253d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:33.960110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7f0f6650-0a7e-4fd7-87c6-f295e8c6253d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:33.991573 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab9bbd-50aa-4700-8a8c-57528ab03e56 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:34.026753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5ce95-4e19-4607-830c-ce8eb8074db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:34.027387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5ce95-4e19-4607-830c-ce8eb8074db0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:34.027662 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5ce95-4e19-4607-830c-ce8eb8074db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:34.027933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5ce95-4e19-4607-830c-ce8eb8074db0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:34.029341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 528/1076] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:34.039172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5ce95-4e19-4607-830c-ce8eb8074db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d9b5ce95-4e19-4607-830c-ce8eb8074db0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:17:34.043940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:34.044932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:34.049403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 549/1077] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:17:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:34.051360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f6650-0a7e-4fd7-87c6-f295e8c6253d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:34.151444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f6650-0a7e-4fd7-87c6-f295e8c6253d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:34.165714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:34.187204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76785c78-46da-420d-bc35-b2f15b80886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:34.187802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76785c78-46da-420d-bc35-b2f15b80886f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:34.188077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76785c78-46da-420d-bc35-b2f15b80886f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:34.188343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76785c78-46da-420d-bc35-b2f15b80886f None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:34.189034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76785c78-46da-420d-bc35-b2f15b80886f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:34.189336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76785c78-46da-420d-bc35-b2f15b80886f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:34.190680 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76785c78-46da-420d-bc35-b2f15b80886f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:34.203553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 529/1078] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:17:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:34.205320 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5ce95-4e19-4607-830c-ce8eb8074db0 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:34.303556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:34.305611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76785c78-46da-420d-bc35-b2f15b80886f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:34.309861 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5ce95-4e19-4607-830c-ce8eb8074db0 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:34.348057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbace0-5be6-4c5a-9b0f-03d1952c31dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:34.348723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbace0-5be6-4c5a-9b0f-03d1952c31dc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:34.349412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbace0-5be6-4c5a-9b0f-03d1952c31dc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:34.349691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbace0-5be6-4c5a-9b0f-03d1952c31dc None admin] PATH_INFO: `/v3/users/678ff473fc9d4f409a8887558437a341` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:34.351196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbace0-5be6-4c5a-9b0f-03d1952c31dc None admin] RBAC: Authorizing `identity:get_user(user_id=678ff473fc9d4f409a8887558437a341)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:34.352987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbace0-5be6-4c5a-9b0f-03d1952c31dc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:34.359349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 550/1079] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:17:34 2022] GET /identity/v3/users/678ff473fc9d4f409a8887558437a341 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:34.361164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbace0-5be6-4c5a-9b0f-03d1952c31dc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:34.426351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76785c78-46da-420d-bc35-b2f15b80886f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:34.463641 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6188786-08a8-4bb0-974d-c2056d0b8907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:34.464265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6188786-08a8-4bb0-974d-c2056d0b8907 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:34.464566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6188786-08a8-4bb0-974d-c2056d0b8907 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:34.464831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6188786-08a8-4bb0-974d-c2056d0b8907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:34.476159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbace0-5be6-4c5a-9b0f-03d1952c31dc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:34.510240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:34.513177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f80e4-84f9-4a24-a369-662e3cf62f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:34.513802 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f80e4-84f9-4a24-a369-662e3cf62f33 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:34.514078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f80e4-84f9-4a24-a369-662e3cf62f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:34.514342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f80e4-84f9-4a24-a369-662e3cf62f33 None admin] PATH_INFO: `/v3/projects/8a13e7576fa646b5a7d29584e6e162c6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:34.515432 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f80e4-84f9-4a24-a369-662e3cf62f33 None admin] RBAC: Authorizing `identity:get_project(project_id=8a13e7576fa646b5a7d29584e6e162c6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:34.516885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f80e4-84f9-4a24-a369-662e3cf62f33 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:34.523781 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 551/1080] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:17:34 2022] GET /identity/v3/projects/8a13e7576fa646b5a7d29584e6e162c6 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:34.524758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f80e4-84f9-4a24-a369-662e3cf62f33 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:34.539987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6188786-08a8-4bb0-974d-c2056d0b8907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:34.541476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6188786-08a8-4bb0-974d-c2056d0b8907 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:34.561864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:34.635175 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f80e4-84f9-4a24-a369-662e3cf62f33 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:34.643261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 530/1081] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 30 19:17:34.649387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6188786-08a8-4bb0-974d-c2056d0b8907 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:34.672563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3369f78-4b3f-40e6-aba1-a2eb475cdb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:34.673227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3369f78-4b3f-40e6-aba1-a2eb475cdb94 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:34.673504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3369f78-4b3f-40e6-aba1-a2eb475cdb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:34.673776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3369f78-4b3f-40e6-aba1-a2eb475cdb94 None admin] PATH_INFO: `/v3/projects/8a13e7576fa646b5a7d29584e6e162c6/users/678ff473fc9d4f409a8887558437a341/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:34.675824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3369f78-4b3f-40e6-aba1-a2eb475cdb94 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a13e7576fa646b5a7d29584e6e162c6, user_id=678ff473fc9d4f409a8887558437a341, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:34.677247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3369f78-4b3f-40e6-aba1-a2eb475cdb94 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:34.691356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 552/1082] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:17:34 2022] PUT /identity/v3/projects/8a13e7576fa646b5a7d29584e6e162c6/users/678ff473fc9d4f409a8887558437a341/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:17:34.731589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:34.760125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6188786-08a8-4bb0-974d-c2056d0b8907 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:34.803120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f587bb-2f0b-4cef-9e00-b51c1366fe45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:34.803797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f587bb-2f0b-4cef-9e00-b51c1366fe45 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:34.804070 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f587bb-2f0b-4cef-9e00-b51c1366fe45 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:34.804335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f587bb-2f0b-4cef-9e00-b51c1366fe45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:34.815713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f587bb-2f0b-4cef-9e00-b51c1366fe45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:34.817218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f587bb-2f0b-4cef-9e00-b51c1366fe45 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-07f587bb-2f0b-4cef-9e00-b51c1366fe45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:34.833024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:34.834180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:34.835735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:34.835735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:34.835735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:34.839838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 531/1083] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:34.844428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3369f78-4b3f-40e6-aba1-a2eb475cdb94 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:34.940681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:34.950221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3369f78-4b3f-40e6-aba1-a2eb475cdb94 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:34.983352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc2cbc-f82d-4a12-b408-cc1fbedbf14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:34.983953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc2cbc-f82d-4a12-b408-cc1fbedbf14d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:34.984225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc2cbc-f82d-4a12-b408-cc1fbedbf14d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:34.984519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc2cbc-f82d-4a12-b408-cc1fbedbf14d None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:34.985147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc2cbc-f82d-4a12-b408-cc1fbedbf14d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:34.985442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc2cbc-f82d-4a12-b408-cc1fbedbf14d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:34.986782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc2cbc-f82d-4a12-b408-cc1fbedbf14d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:34.998892 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 553/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:35.779142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6dba6-1268-49cd-9bad-67edfe21c858 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:35.779697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6dba6-1268-49cd-9bad-67edfe21c858 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:35.779697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6dba6-1268-49cd-9bad-67edfe21c858 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:35.781023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 532/1085] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:35.786206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028c159-3ee1-410f-8cfa-c828144197da None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:35.786550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028c159-3ee1-410f-8cfa-c828144197da None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:35.786885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028c159-3ee1-410f-8cfa-c828144197da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:35.805470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3028c159-3ee1-410f-8cfa-c828144197da None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:35.806628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3028c159-3ee1-410f-8cfa-c828144197da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:35.873524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 554/1086] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:36.148140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6dba6-1268-49cd-9bad-67edfe21c858 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:36.252195 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6dba6-1268-49cd-9bad-67edfe21c858 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:36.277510 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:36.286169 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e44f5-837a-40c9-9952-c4bc5970fe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:36.286791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e44f5-837a-40c9-9952-c4bc5970fe40 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:36.287065 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e44f5-837a-40c9-9952-c4bc5970fe40 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:36.287332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e44f5-837a-40c9-9952-c4bc5970fe40 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:36.297312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e44f5-837a-40c9-9952-c4bc5970fe40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b19e44f5-837a-40c9-9952-c4bc5970fe40 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:17:36.302010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:36.302995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:36.304497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 533/1087] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Sep 30 19:17:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:36.309666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028c159-3ee1-410f-8cfa-c828144197da None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:36.390005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:36.407343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028c159-3ee1-410f-8cfa-c828144197da None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:36.440884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b672c7-2a73-48f3-b5b9-431e41c0f494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:36.441481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b672c7-2a73-48f3-b5b9-431e41c0f494 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:36.441751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b672c7-2a73-48f3-b5b9-431e41c0f494 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:36.442014 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b672c7-2a73-48f3-b5b9-431e41c0f494 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:36.442664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b672c7-2a73-48f3-b5b9-431e41c0f494 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:36.442958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b672c7-2a73-48f3-b5b9-431e41c0f494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:36.444883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b672c7-2a73-48f3-b5b9-431e41c0f494 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:36.452656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 555/1088] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 30 19:17:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:36.458387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e44f5-837a-40c9-9952-c4bc5970fe40 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:36.556631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e44f5-837a-40c9-9952-c4bc5970fe40 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:36.591087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525e0f-ff59-4dfb-9829-aab51a5d02c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:36.591704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525e0f-ff59-4dfb-9829-aab51a5d02c0 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:36.591977 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525e0f-ff59-4dfb-9829-aab51a5d02c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:36.592240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a525e0f-ff59-4dfb-9829-aab51a5d02c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:36.601978 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525e0f-ff59-4dfb-9829-aab51a5d02c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:36.603399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a525e0f-ff59-4dfb-9829-aab51a5d02c0 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-6a525e0f-ff59-4dfb-9829-aab51a5d02c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:36.617240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:36.623020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:36.624079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:36.624079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:36.624079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:36.624079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 534/1089] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:36.624313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b672c7-2a73-48f3-b5b9-431e41c0f494 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:36.722821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b672c7-2a73-48f3-b5b9-431e41c0f494 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:36.759469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a65ef9-8e8c-4e84-8d4f-c977c697ef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:36.760085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65ef9-8e8c-4e84-8d4f-c977c697ef50 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:36.760360 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65ef9-8e8c-4e84-8d4f-c977c697ef50 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:36.760651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65ef9-8e8c-4e84-8d4f-c977c697ef50 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:36.761288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a65ef9-8e8c-4e84-8d4f-c977c697ef50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:36.761584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a65ef9-8e8c-4e84-8d4f-c977c697ef50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:36.762932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a65ef9-8e8c-4e84-8d4f-c977c697ef50 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:36.774606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 556/1090] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:36.780384 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525e0f-ff59-4dfb-9829-aab51a5d02c0 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:36.818911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:36.844894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56cbd5-4102-4d1a-b6de-507229e41243 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:36.845176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56cbd5-4102-4d1a-b6de-507229e41243 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:36.845448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56cbd5-4102-4d1a-b6de-507229e41243 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:36.846834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 557/1091] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:17:36.851817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb907c-7750-443f-b0d7-a04893490d8f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:36.852098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb907c-7750-443f-b0d7-a04893490d8f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:36.852385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb907c-7750-443f-b0d7-a04893490d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:36.864012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:36.870564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-40cb907c-7750-443f-b0d7-a04893490d8f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:36.871755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-40cb907c-7750-443f-b0d7-a04893490d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:36.879887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a525e0f-ff59-4dfb-9829-aab51a5d02c0 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:36.913103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0a55d-d885-44ba-ab20-3e358a181170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:36.913854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0a55d-d885-44ba-ab20-3e358a181170 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:36.914132 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0a55d-d885-44ba-ab20-3e358a181170 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:36.914399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0a55d-d885-44ba-ab20-3e358a181170 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:36.924143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0a55d-d885-44ba-ab20-3e358a181170 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:36.925671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0a55d-d885-44ba-ab20-3e358a181170 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:36.937075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 558/1092] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-b2f0a55d-d885-44ba-ab20-3e358a181170 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:17:36.937455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:36.938600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:36.939618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:36.939618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:36.939618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:17:36.939618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:17:36.939618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:17:36.939618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:17:36.939618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:17:36.939618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:17:36.939618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 19:17:36.939618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:36.939618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 535/1093] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Sep 30 19:17:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:36.944693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb907c-7750-443f-b0d7-a04893490d8f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:37.040860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb907c-7750-443f-b0d7-a04893490d8f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:37.073867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eee1a-2522-40b1-bd04-9a06dc2acd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:37.074460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eee1a-2522-40b1-bd04-9a06dc2acd41 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:37.074732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eee1a-2522-40b1-bd04-9a06dc2acd41 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:37.075000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134eee1a-2522-40b1-bd04-9a06dc2acd41 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:37.075706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eee1a-2522-40b1-bd04-9a06dc2acd41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:37.076003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eee1a-2522-40b1-bd04-9a06dc2acd41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:37.077432 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134eee1a-2522-40b1-bd04-9a06dc2acd41 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:37.079908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:37.095016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 559/1094] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Sep 30 19:17:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:37.101328 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0a55d-d885-44ba-ab20-3e358a181170 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:37.193972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:37.198227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0a55d-d885-44ba-ab20-3e358a181170 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:37.208488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eee1a-2522-40b1-bd04-9a06dc2acd41 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:37.232836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0c4a5-7481-49aa-8804-731594b91acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:37.233435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0c4a5-7481-49aa-8804-731594b91acd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:37.233710 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0c4a5-7481-49aa-8804-731594b91acd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:37.233975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0c4a5-7481-49aa-8804-731594b91acd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:37.303011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0c4a5-7481-49aa-8804-731594b91acd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:37.304388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0c4a5-7481-49aa-8804-731594b91acd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:37.306970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134eee1a-2522-40b1-bd04-9a06dc2acd41 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:37.341289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97450188-3f03-4b57-9e7e-751b926c1717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:37.341986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450188-3f03-4b57-9e7e-751b926c1717 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:37.342269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450188-3f03-4b57-9e7e-751b926c1717 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:37.342540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450188-3f03-4b57-9e7e-751b926c1717 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:37.352406 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97450188-3f03-4b57-9e7e-751b926c1717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-97450188-3f03-4b57-9e7e-751b926c1717 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:17:37.357160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:37.360426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:17:37.361574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 560/1095] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:17:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:37.364240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97450188-3f03-4b57-9e7e-751b926c1717 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:37.407027 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:37.411515 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 536/1096] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:17:37.417239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0c4a5-7481-49aa-8804-731594b91acd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:37.419044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:37.462557 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97450188-3f03-4b57-9e7e-751b926c1717 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:37.496229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64807520-17d8-4f33-b11d-842e2f168186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:37.496860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64807520-17d8-4f33-b11d-842e2f168186 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:37.497132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64807520-17d8-4f33-b11d-842e2f168186 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:37.497397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64807520-17d8-4f33-b11d-842e2f168186 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:37.498047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64807520-17d8-4f33-b11d-842e2f168186 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:37.498341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64807520-17d8-4f33-b11d-842e2f168186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:37.499677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64807520-17d8-4f33-b11d-842e2f168186 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:37.507512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 561/1097] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:37.513065 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0c4a5-7481-49aa-8804-731594b91acd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:37.513396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64807520-17d8-4f33-b11d-842e2f168186 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:37.548763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfb23f-3f72-4e7b-badd-d37f9cb9564e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:37.549384 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfb23f-3f72-4e7b-badd-d37f9cb9564e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:37.549659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfb23f-3f72-4e7b-badd-d37f9cb9564e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:37.549928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfb23f-3f72-4e7b-badd-d37f9cb9564e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:37.560092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfb23f-3f72-4e7b-badd-d37f9cb9564e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:37.561506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfb23f-3f72-4e7b-badd-d37f9cb9564e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-cdbfb23f-3f72-4e7b-badd-d37f9cb9564e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:17:37.575904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:37.577077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:17:37.578223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:17:37.578223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:17:37.578223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:37.578223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 537/1098] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:37.583980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfb23f-3f72-4e7b-badd-d37f9cb9564e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:37.611066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64807520-17d8-4f33-b11d-842e2f168186 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:37.628583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:37.633973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:37.647152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efe26a-e7c6-45bd-89ae-c9dc61f6359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:37.647775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efe26a-e7c6-45bd-89ae-c9dc61f6359f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:37.648054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efe26a-e7c6-45bd-89ae-c9dc61f6359f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:37.648325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efe26a-e7c6-45bd-89ae-c9dc61f6359f None admin] PATH_INFO: `/v3/users/678ff473fc9d4f409a8887558437a341` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:37.649470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3efe26a-e7c6-45bd-89ae-c9dc61f6359f None admin] RBAC: Authorizing `identity:get_user(user_id=678ff473fc9d4f409a8887558437a341)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:37.650884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3efe26a-e7c6-45bd-89ae-c9dc61f6359f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:37.652458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 562/1099] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:17:37 2022] GET /identity/v3/users/678ff473fc9d4f409a8887558437a341 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:37.658235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efe26a-e7c6-45bd-89ae-c9dc61f6359f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:37.683213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfb23f-3f72-4e7b-badd-d37f9cb9564e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:37.715387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b85cf-6c64-411c-841a-b2200cb3fed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:37.715978 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b85cf-6c64-411c-841a-b2200cb3fed9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:37.716249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b85cf-6c64-411c-841a-b2200cb3fed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:37.716548 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b85cf-6c64-411c-841a-b2200cb3fed9 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:37.717175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b85cf-6c64-411c-841a-b2200cb3fed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:37.717473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b85cf-6c64-411c-841a-b2200cb3fed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:37.718800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b85cf-6c64-411c-841a-b2200cb3fed9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:37.730698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 538/1100] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:37.736298 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b85cf-6c64-411c-841a-b2200cb3fed9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:37.753755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efe26a-e7c6-45bd-89ae-c9dc61f6359f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:37.787093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06ed8f-f266-4ad9-ba1a-13d86225e294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:37.788542 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06ed8f-f266-4ad9-ba1a-13d86225e294 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:37.788795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06ed8f-f266-4ad9-ba1a-13d86225e294 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:37.789064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06ed8f-f266-4ad9-ba1a-13d86225e294 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:37.833686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b85cf-6c64-411c-841a-b2200cb3fed9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:37.838330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:37.844817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:37.856170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06ed8f-f266-4ad9-ba1a-13d86225e294 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:37.857661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06ed8f-f266-4ad9-ba1a-13d86225e294 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:37.869415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2696e5-a27d-465c-8102-c65e7592986e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:37.870042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2696e5-a27d-465c-8102-c65e7592986e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:37.870316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2696e5-a27d-465c-8102-c65e7592986e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:37.870589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2696e5-a27d-465c-8102-c65e7592986e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:37.877678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2696e5-a27d-465c-8102-c65e7592986e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:37.879105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2696e5-a27d-465c-8102-c65e7592986e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-5a2696e5-a27d-465c-8102-c65e7592986e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:17:37.890529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:37.891464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:37.892410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:17:37.892410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:17:37.892410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:17:37.892410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:17:37.892410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:17:37.892410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:17:37.892410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:17:37.892410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:37.892410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 539/1101] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:17:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:17:37.898125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2696e5-a27d-465c-8102-c65e7592986e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:37.963167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 563/1102] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 30 19:17:37.965369 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06ed8f-f266-4ad9-ba1a-13d86225e294 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:37.998798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2696e5-a27d-465c-8102-c65e7592986e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:38.031192 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea6cea-cb9f-4ae3-8b3a-6d29a860c96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:38.031780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea6cea-cb9f-4ae3-8b3a-6d29a860c96c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:38.032058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea6cea-cb9f-4ae3-8b3a-6d29a860c96c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:38.032322 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea6cea-cb9f-4ae3-8b3a-6d29a860c96c None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:38.033020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea6cea-cb9f-4ae3-8b3a-6d29a860c96c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:38.033316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea6cea-cb9f-4ae3-8b3a-6d29a860c96c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:38.034721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea6cea-cb9f-4ae3-8b3a-6d29a860c96c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:38.043574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:38.047697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 540/1103] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:17:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:38.053851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea6cea-cb9f-4ae3-8b3a-6d29a860c96c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:38.059752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06ed8f-f266-4ad9-ba1a-13d86225e294 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:38.084309 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:38.099043 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfdcc5-5b88-49bc-be28-3ca98078cce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:38.099665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfdcc5-5b88-49bc-be28-3ca98078cce3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:38.099936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfdcc5-5b88-49bc-be28-3ca98078cce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:38.100744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfdcc5-5b88-49bc-be28-3ca98078cce3 None admin] PATH_INFO: `/v3/projects/8a13e7576fa646b5a7d29584e6e162c6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:38.101829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfdcc5-5b88-49bc-be28-3ca98078cce3 None admin] RBAC: Authorizing `identity:get_project(project_id=8a13e7576fa646b5a7d29584e6e162c6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:38.103237 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfdcc5-5b88-49bc-be28-3ca98078cce3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:38.104870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 564/1104] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:17:37 2022] GET /identity/v3/projects/8a13e7576fa646b5a7d29584e6e162c6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:38.110904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfdcc5-5b88-49bc-be28-3ca98078cce3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:38.156759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea6cea-cb9f-4ae3-8b3a-6d29a860c96c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:38.190258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b9fbe-d51e-4207-a2dd-1cff4442b8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:38.190868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b9fbe-d51e-4207-a2dd-1cff4442b8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:38.191144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b9fbe-d51e-4207-a2dd-1cff4442b8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:38.191415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b9fbe-d51e-4207-a2dd-1cff4442b8d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:38.192142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b9fbe-d51e-4207-a2dd-1cff4442b8d7 None admin] RBAC: Adding query filter params (role.id=657af95d19214896a42b23c30ab7a220, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=5952db29e6fc4ddbb9aeb0522497312c) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:38.192476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b9fbe-d51e-4207-a2dd-1cff4442b8d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:38.194391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b9fbe-d51e-4207-a2dd-1cff4442b8d7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:38.200289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 541/1105] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:17:38 2022] GET /identity/v3/role_assignments?user.id=5952db29e6fc4ddbb9aeb0522497312c&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=657af95d19214896a42b23c30ab7a220 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:38.208317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfdcc5-5b88-49bc-be28-3ca98078cce3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:38.247469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459801d6-c09f-40fe-9ff1-509ef42fa70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:38.248079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459801d6-c09f-40fe-9ff1-509ef42fa70d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:38.248353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459801d6-c09f-40fe-9ff1-509ef42fa70d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:38.248649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459801d6-c09f-40fe-9ff1-509ef42fa70d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:38.249378 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459801d6-c09f-40fe-9ff1-509ef42fa70d None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=8a13e7576fa646b5a7d29584e6e162c6, user.id=678ff473fc9d4f409a8887558437a341) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:38.249681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459801d6-c09f-40fe-9ff1-509ef42fa70d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:38.251119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459801d6-c09f-40fe-9ff1-509ef42fa70d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:38.258222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 565/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:17:38 2022] GET /identity/v3/role_assignments?user.id=678ff473fc9d4f409a8887558437a341&scope.project.id=8a13e7576fa646b5a7d29584e6e162c6&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:40.419725 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed2662-79bf-4bdd-9b18-6efab1f1f137 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:40.420162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed2662-79bf-4bdd-9b18-6efab1f1f137 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:40.420275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed2662-79bf-4bdd-9b18-6efab1f1f137 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:40.421688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 542/1107] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:40.426996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309de434-0681-4348-813f-c51dd45ec11a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:40.427394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309de434-0681-4348-813f-c51dd45ec11a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:40.427529 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309de434-0681-4348-813f-c51dd45ec11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:40.445896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-309de434-0681-4348-813f-c51dd45ec11a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:40.447063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-309de434-0681-4348-813f-c51dd45ec11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:40.457225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:40.512627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 566/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:40.779034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed2662-79bf-4bdd-9b18-6efab1f1f137 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:40.871275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:40.878919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed2662-79bf-4bdd-9b18-6efab1f1f137 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:40.926303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727d3a6-8053-4242-a00d-9bf366428ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:40.926926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d3a6-8053-4242-a00d-9bf366428ebb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:40.927201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d3a6-8053-4242-a00d-9bf366428ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:40.927467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727d3a6-8053-4242-a00d-9bf366428ebb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:40.929237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727d3a6-8053-4242-a00d-9bf366428ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:40.930672 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727d3a6-8053-4242-a00d-9bf366428ebb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:40.932271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 543/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:40.938303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309de434-0681-4348-813f-c51dd45ec11a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:41.029797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309de434-0681-4348-813f-c51dd45ec11a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:41.061928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eabdd0-7f2c-4c91-a255-b67688df6890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:41.062519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eabdd0-7f2c-4c91-a255-b67688df6890 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:41.062819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eabdd0-7f2c-4c91-a255-b67688df6890 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:41.063097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eabdd0-7f2c-4c91-a255-b67688df6890 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:41.063762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eabdd0-7f2c-4c91-a255-b67688df6890 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:41.065209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eabdd0-7f2c-4c91-a255-b67688df6890 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:41.086112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 567/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:17:40 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:17:43.235998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5e99f-a3cf-409e-9c1a-9e035fbf80e7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:43.236589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5e99f-a3cf-409e-9c1a-9e035fbf80e7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:43.236589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5e99f-a3cf-409e-9c1a-9e035fbf80e7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:43.237873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 544/1111] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:43.243194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012c709-7af1-47cf-8fbf-13afeba18fb4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:43.243567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012c709-7af1-47cf-8fbf-13afeba18fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:43.243729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012c709-7af1-47cf-8fbf-13afeba18fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:43.249951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:43.263199 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5012c709-7af1-47cf-8fbf-13afeba18fb4 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:43.264360 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5012c709-7af1-47cf-8fbf-13afeba18fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:43.327520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 568/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:43.595612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5e99f-a3cf-409e-9c1a-9e035fbf80e7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:43.692468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5e99f-a3cf-409e-9c1a-9e035fbf80e7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:43.724771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fe37f-ea92-4caa-a389-4d91d1be7896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:43.726009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fe37f-ea92-4caa-a389-4d91d1be7896 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:43.726278 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fe37f-ea92-4caa-a389-4d91d1be7896 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:43.726543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fe37f-ea92-4caa-a389-4d91d1be7896 None admin] PATH_INFO: `/v3/roles/9c17da6cc0e54b67af7cb7fb68638a7e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:43.727920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fe37f-ea92-4caa-a389-4d91d1be7896 None admin] RBAC: Authorizing `identity:get_role(role_id=9c17da6cc0e54b67af7cb7fb68638a7e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:43.729304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fe37f-ea92-4caa-a389-4d91d1be7896 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:43.730593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 545/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:17:43 2022] GET /identity/v3/roles/9c17da6cc0e54b67af7cb7fb68638a7e => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:43.736181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012c709-7af1-47cf-8fbf-13afeba18fb4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:43.828119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012c709-7af1-47cf-8fbf-13afeba18fb4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:43.856509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:43.861530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca33961-6080-4cdf-b728-d2678829669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:43.862145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33961-6080-4cdf-b728-d2678829669e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:43.862422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33961-6080-4cdf-b728-d2678829669e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:43.862691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33961-6080-4cdf-b728-d2678829669e None admin] PATH_INFO: `/v3/users/fe4045fb269a40249e6d665605de6533` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:43.875115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca33961-6080-4cdf-b728-d2678829669e None admin] RBAC: Authorizing `identity:get_user(user_id=fe4045fb269a40249e6d665605de6533)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:43.876562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca33961-6080-4cdf-b728-d2678829669e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:43.878118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 569/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:17:43 2022] GET /identity/v3/users/fe4045fb269a40249e6d665605de6533 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:17:43.884312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fe37f-ea92-4caa-a389-4d91d1be7896 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:43.914261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:43.980672 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fe37f-ea92-4caa-a389-4d91d1be7896 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:44.012639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772e2f8-45f1-44ae-b406-6a3ec23ca0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:44.013222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772e2f8-45f1-44ae-b406-6a3ec23ca0cf None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:44.013503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772e2f8-45f1-44ae-b406-6a3ec23ca0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:44.013769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772e2f8-45f1-44ae-b406-6a3ec23ca0cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:44.079902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772e2f8-45f1-44ae-b406-6a3ec23ca0cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:44.081386 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772e2f8-45f1-44ae-b406-6a3ec23ca0cf None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:44.109626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:44.181327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 546/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:17:44.187263 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca33961-6080-4cdf-b728-d2678829669e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:44.281364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca33961-6080-4cdf-b728-d2678829669e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:44.313987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abd224-8112-44d0-bf04-d548fd7fe6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:44.314600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abd224-8112-44d0-bf04-d548fd7fe6eb None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:44.314878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abd224-8112-44d0-bf04-d548fd7fe6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:44.315147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abd224-8112-44d0-bf04-d548fd7fe6eb None admin] PATH_INFO: `/v3/projects/8a13e7576fa646b5a7d29584e6e162c6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:44.316230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abd224-8112-44d0-bf04-d548fd7fe6eb None admin] RBAC: Authorizing `identity:get_project(project_id=8a13e7576fa646b5a7d29584e6e162c6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:44.317665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abd224-8112-44d0-bf04-d548fd7fe6eb None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:44.319306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 570/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:17:44 2022] GET /identity/v3/projects/8a13e7576fa646b5a7d29584e6e162c6 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:44.325294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772e2f8-45f1-44ae-b406-6a3ec23ca0cf None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:44.420340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772e2f8-45f1-44ae-b406-6a3ec23ca0cf None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:44.447847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:44.453830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909944f-b551-4f02-8016-dbff11ffa2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:44.454426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909944f-b551-4f02-8016-dbff11ffa2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:44.454702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909944f-b551-4f02-8016-dbff11ffa2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:44.454972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909944f-b551-4f02-8016-dbff11ffa2f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:44.455686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909944f-b551-4f02-8016-dbff11ffa2f4 None admin] RBAC: Adding query filter params (role.id=9c17da6cc0e54b67af7cb7fb68638a7e, scope.project.id=8a13e7576fa646b5a7d29584e6e162c6, user.id=fe4045fb269a40249e6d665605de6533) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:44.455992 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909944f-b551-4f02-8016-dbff11ffa2f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:44.457573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909944f-b551-4f02-8016-dbff11ffa2f4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:44.463304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 547/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:17:44 2022] GET /identity/v3/role_assignments?user.id=fe4045fb269a40249e6d665605de6533&scope.project.id=8a13e7576fa646b5a7d29584e6e162c6&role.id=9c17da6cc0e54b67af7cb7fb68638a7e => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:46.556008 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13d3bd-5e20-4aff-853f-8f6e1f49738d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:46.556555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13d3bd-5e20-4aff-853f-8f6e1f49738d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:46.556555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13d3bd-5e20-4aff-853f-8f6e1f49738d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:46.557890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 571/1118] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:46.563022 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ccaf0-9da7-455c-8b2a-58522ba340f5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:46.563440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ccaf0-9da7-455c-8b2a-58522ba340f5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:46.563568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ccaf0-9da7-455c-8b2a-58522ba340f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:46.581776 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c66ccaf0-9da7-455c-8b2a-58522ba340f5 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:46.582943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c66ccaf0-9da7-455c-8b2a-58522ba340f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:46.646130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 548/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:46.914986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13d3bd-5e20-4aff-853f-8f6e1f49738d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:46.971988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:47.013432 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13d3bd-5e20-4aff-853f-8f6e1f49738d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:47.045505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe743d71-eed6-4a24-8b66-6387d772192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:47.046116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe743d71-eed6-4a24-8b66-6387d772192f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:47.046388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe743d71-eed6-4a24-8b66-6387d772192f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:47.046657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe743d71-eed6-4a24-8b66-6387d772192f None admin] PATH_INFO: `/v3/roles/9c17da6cc0e54b67af7cb7fb68638a7e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:47.047996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe743d71-eed6-4a24-8b66-6387d772192f None admin] RBAC: Authorizing `identity:get_role(role_id=9c17da6cc0e54b67af7cb7fb68638a7e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:47.049383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe743d71-eed6-4a24-8b66-6387d772192f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:47.050667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 572/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:17:46 2022] GET /identity/v3/roles/9c17da6cc0e54b67af7cb7fb68638a7e => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:47.056910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ccaf0-9da7-455c-8b2a-58522ba340f5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:47.149668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ccaf0-9da7-455c-8b2a-58522ba340f5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:47.172223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:47.183049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9418e4-7c80-4f28-8d7f-64ed64fe7b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:47.183657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9418e4-7c80-4f28-8d7f-64ed64fe7b52 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:47.183932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9418e4-7c80-4f28-8d7f-64ed64fe7b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:47.184199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9418e4-7c80-4f28-8d7f-64ed64fe7b52 None admin] PATH_INFO: `/v3/users/fe4045fb269a40249e6d665605de6533` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:47.185316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9418e4-7c80-4f28-8d7f-64ed64fe7b52 None admin] RBAC: Authorizing `identity:get_user(user_id=fe4045fb269a40249e6d665605de6533)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:47.186735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9418e4-7c80-4f28-8d7f-64ed64fe7b52 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:47.188262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 549/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:17:47 2022] GET /identity/v3/users/fe4045fb269a40249e6d665605de6533 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:47.193815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe743d71-eed6-4a24-8b66-6387d772192f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:47.288735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe743d71-eed6-4a24-8b66-6387d772192f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:47.312352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:47.320893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eac916-46ea-4a38-94d0-659b0ed6b737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:47.321505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eac916-46ea-4a38-94d0-659b0ed6b737 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:47.321779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eac916-46ea-4a38-94d0-659b0ed6b737 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:47.322048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eac916-46ea-4a38-94d0-659b0ed6b737 None admin] PATH_INFO: `/v3/projects/8a13e7576fa646b5a7d29584e6e162c6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:47.323141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eac916-46ea-4a38-94d0-659b0ed6b737 None admin] RBAC: Authorizing `identity:get_project(project_id=8a13e7576fa646b5a7d29584e6e162c6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:47.324586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eac916-46ea-4a38-94d0-659b0ed6b737 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:47.326171 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 573/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:17:47 2022] GET /identity/v3/projects/8a13e7576fa646b5a7d29584e6e162c6 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:47.332108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9418e4-7c80-4f28-8d7f-64ed64fe7b52 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:47.428785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9418e4-7c80-4f28-8d7f-64ed64fe7b52 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:47.460887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be236c-6dce-45ea-97d8-a43169d45608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:47.461529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be236c-6dce-45ea-97d8-a43169d45608 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:47.461805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be236c-6dce-45ea-97d8-a43169d45608 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:47.462074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be236c-6dce-45ea-97d8-a43169d45608 None admin] PATH_INFO: `/v3/projects/8a13e7576fa646b5a7d29584e6e162c6/users/fe4045fb269a40249e6d665605de6533/roles/9c17da6cc0e54b67af7cb7fb68638a7e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:47.464211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be236c-6dce-45ea-97d8-a43169d45608 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a13e7576fa646b5a7d29584e6e162c6, user_id=fe4045fb269a40249e6d665605de6533, role_id=9c17da6cc0e54b67af7cb7fb68638a7e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:47.465655 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be236c-6dce-45ea-97d8-a43169d45608 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:47.478476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 550/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:17:47 2022] PUT /identity/v3/projects/8a13e7576fa646b5a7d29584e6e162c6/users/fe4045fb269a40249e6d665605de6533/roles/9c17da6cc0e54b67af7cb7fb68638a7e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:17:49.559807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938e4c3-d2ce-474b-a488-b77aa82f97d7 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:49.560351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938e4c3-d2ce-474b-a488-b77aa82f97d7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:49.560351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938e4c3-d2ce-474b-a488-b77aa82f97d7 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:49.561704 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 574/1124] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:49.566956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251c31b-ff19-427d-8304-7cc738701936 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:49.567393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251c31b-ff19-427d-8304-7cc738701936 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:49.567500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251c31b-ff19-427d-8304-7cc738701936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:49.585848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d251c31b-ff19-427d-8304-7cc738701936 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:49.586996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d251c31b-ff19-427d-8304-7cc738701936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:49.631710 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:49.651417 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 551/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:49.916882 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938e4c3-d2ce-474b-a488-b77aa82f97d7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:50.013457 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938e4c3-d2ce-474b-a488-b77aa82f97d7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:50.045796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273eff1-7c3b-4867-af4f-921b35f2d3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:50.046401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273eff1-7c3b-4867-af4f-921b35f2d3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:50.046674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273eff1-7c3b-4867-af4f-921b35f2d3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:50.046940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273eff1-7c3b-4867-af4f-921b35f2d3d6 None admin] PATH_INFO: `/v3/roles/9c17da6cc0e54b67af7cb7fb68638a7e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:50.048307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273eff1-7c3b-4867-af4f-921b35f2d3d6 None admin] RBAC: Authorizing `identity:get_role(role_id=9c17da6cc0e54b67af7cb7fb68638a7e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:50.049662 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273eff1-7c3b-4867-af4f-921b35f2d3d6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:50.050931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 575/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:17:49 2022] GET /identity/v3/roles/9c17da6cc0e54b67af7cb7fb68638a7e => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:50.057103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251c31b-ff19-427d-8304-7cc738701936 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:50.151740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251c31b-ff19-427d-8304-7cc738701936 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:50.186331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842ba44-7576-457a-8082-4ff4c9ab39fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:50.186948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842ba44-7576-457a-8082-4ff4c9ab39fa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:50.187224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842ba44-7576-457a-8082-4ff4c9ab39fa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:50.187494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842ba44-7576-457a-8082-4ff4c9ab39fa None admin] PATH_INFO: `/v3/users/fe4045fb269a40249e6d665605de6533` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:50.188610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842ba44-7576-457a-8082-4ff4c9ab39fa None admin] RBAC: Authorizing `identity:get_user(user_id=fe4045fb269a40249e6d665605de6533)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:50.190036 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842ba44-7576-457a-8082-4ff4c9ab39fa None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:50.191580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 552/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:17:50 2022] GET /identity/v3/users/fe4045fb269a40249e6d665605de6533 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:50.197382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273eff1-7c3b-4867-af4f-921b35f2d3d6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:50.255517 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:50.292569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273eff1-7c3b-4867-af4f-921b35f2d3d6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:50.323646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a815a-6efe-411c-a6f6-1cf91bcd0533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:50.324198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a815a-6efe-411c-a6f6-1cf91bcd0533 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:50.324498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a815a-6efe-411c-a6f6-1cf91bcd0533 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:50.324767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a815a-6efe-411c-a6f6-1cf91bcd0533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:50.388711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a815a-6efe-411c-a6f6-1cf91bcd0533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:50.390155 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a815a-6efe-411c-a6f6-1cf91bcd0533 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:50.448933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:50.486485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 576/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 30 19:17:50.492962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842ba44-7576-457a-8082-4ff4c9ab39fa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:50.540857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:50.588701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842ba44-7576-457a-8082-4ff4c9ab39fa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:50.621740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e72907-da9b-41e7-a79d-2745c7e57630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:50.622349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72907-da9b-41e7-a79d-2745c7e57630 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:50.622626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72907-da9b-41e7-a79d-2745c7e57630 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:50.622895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72907-da9b-41e7-a79d-2745c7e57630 None admin] PATH_INFO: `/v3/projects/8a13e7576fa646b5a7d29584e6e162c6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:50.623981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e72907-da9b-41e7-a79d-2745c7e57630 None admin] RBAC: Authorizing `identity:get_project(project_id=8a13e7576fa646b5a7d29584e6e162c6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:50.625430 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e72907-da9b-41e7-a79d-2745c7e57630 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:50.627017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 553/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:17:50 2022] GET /identity/v3/projects/8a13e7576fa646b5a7d29584e6e162c6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:50.633264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a815a-6efe-411c-a6f6-1cf91bcd0533 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:50.726342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a815a-6efe-411c-a6f6-1cf91bcd0533 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:50.758384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768c1c6-800d-41b5-b561-6cebe1df9dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:50.758962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768c1c6-800d-41b5-b561-6cebe1df9dca None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:50.759237 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768c1c6-800d-41b5-b561-6cebe1df9dca None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:50.759504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768c1c6-800d-41b5-b561-6cebe1df9dca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:50.760210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768c1c6-800d-41b5-b561-6cebe1df9dca None admin] RBAC: Adding query filter params (role.id=9c17da6cc0e54b67af7cb7fb68638a7e, scope.project.id=8a13e7576fa646b5a7d29584e6e162c6, user.id=fe4045fb269a40249e6d665605de6533) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:50.760540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768c1c6-800d-41b5-b561-6cebe1df9dca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:50.761945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768c1c6-800d-41b5-b561-6cebe1df9dca None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:50.767355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 577/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:17:50 2022] GET /identity/v3/role_assignments?user.id=fe4045fb269a40249e6d665605de6533&scope.project.id=8a13e7576fa646b5a7d29584e6e162c6&role.id=9c17da6cc0e54b67af7cb7fb68638a7e => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:52.890361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba24ff-d38f-4c0d-b7c7-425067a7ed4e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:52.890887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba24ff-d38f-4c0d-b7c7-425067a7ed4e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:52.890887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba24ff-d38f-4c0d-b7c7-425067a7ed4e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:52.892225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 554/1131] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:52.898091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0aa36-5c05-4612-9aba-d701c1c6066c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:52.898481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0aa36-5c05-4612-9aba-d701c1c6066c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:52.898627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0aa36-5c05-4612-9aba-d701c1c6066c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:52.916407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ead0aa36-5c05-4612-9aba-d701c1c6066c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:52.917562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ead0aa36-5c05-4612-9aba-d701c1c6066c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:52.936194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:52.981044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 578/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:53.244194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba24ff-d38f-4c0d-b7c7-425067a7ed4e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:53.343196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba24ff-d38f-4c0d-b7c7-425067a7ed4e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:53.360401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:53.376588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8d00e-e0ec-4062-b29c-51fb3d117ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:53.377201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8d00e-e0ec-4062-b29c-51fb3d117ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:53.377475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8d00e-e0ec-4062-b29c-51fb3d117ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:53.377739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8d00e-e0ec-4062-b29c-51fb3d117ce4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:53.378823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8d00e-e0ec-4062-b29c-51fb3d117ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:53.380243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8d00e-e0ec-4062-b29c-51fb3d117ce4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:53.381840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 555/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:53.387681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0aa36-5c05-4612-9aba-d701c1c6066c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:53.484662 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0aa36-5c05-4612-9aba-d701c1c6066c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:53.517176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc14a5-2a94-47a8-a87b-4ca7a243345f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:53.517715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc14a5-2a94-47a8-a87b-4ca7a243345f None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:53.517987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc14a5-2a94-47a8-a87b-4ca7a243345f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:53.518253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc14a5-2a94-47a8-a87b-4ca7a243345f None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:53.518878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc14a5-2a94-47a8-a87b-4ca7a243345f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:53.520270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc14a5-2a94-47a8-a87b-4ca7a243345f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:53.545484 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 579/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:17:53 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:17:55.669299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5135c8-6d12-4660-802b-6ae5df94a017 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:55.669813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5135c8-6d12-4660-802b-6ae5df94a017 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:55.669813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5135c8-6d12-4660-802b-6ae5df94a017 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:55.671096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 556/1135] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:55.676217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c04a57-d228-49b9-aeb1-caad74d14111 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:55.676653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c04a57-d228-49b9-aeb1-caad74d14111 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:55.677107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c04a57-d228-49b9-aeb1-caad74d14111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:55.695851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-72c04a57-d228-49b9-aeb1-caad74d14111 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:55.697047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-72c04a57-d228-49b9-aeb1-caad74d14111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:55.698939 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:55.761469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 580/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:56.027336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5135c8-6d12-4660-802b-6ae5df94a017 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:56.125535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5135c8-6d12-4660-802b-6ae5df94a017 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:56.157223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12c68f-dba3-455c-9633-eb0229081702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:56.157828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12c68f-dba3-455c-9633-eb0229081702 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:56.158099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12c68f-dba3-455c-9633-eb0229081702 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:56.158361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12c68f-dba3-455c-9633-eb0229081702 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:56.159438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12c68f-dba3-455c-9633-eb0229081702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:56.160882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12c68f-dba3-455c-9633-eb0229081702 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:56.163136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 557/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:17:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:56.168974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c04a57-d228-49b9-aeb1-caad74d14111 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:56.266869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c04a57-d228-49b9-aeb1-caad74d14111 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:56.299798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027ecef-dd1b-4da0-961e-b525dd7b229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:56.300333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027ecef-dd1b-4da0-961e-b525dd7b229a None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:56.300641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027ecef-dd1b-4da0-961e-b525dd7b229a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:56.300915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027ecef-dd1b-4da0-961e-b525dd7b229a None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:56.301581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027ecef-dd1b-4da0-961e-b525dd7b229a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:56.302994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027ecef-dd1b-4da0-961e-b525dd7b229a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:56.312647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:56.324128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 581/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:17:56 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:17:58.500723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2588f-9217-462b-9058-5b6c293df1ed None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:58.501241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2588f-9217-462b-9058-5b6c293df1ed None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:58.501241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2588f-9217-462b-9058-5b6c293df1ed None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:58.502593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 558/1139] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:17:58.507942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d384992-c2b1-47e6-bcae-b31e54509a69 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:58.508316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d384992-c2b1-47e6-bcae-b31e54509a69 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:58.508533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d384992-c2b1-47e6-bcae-b31e54509a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:58.526711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9d384992-c2b1-47e6-bcae-b31e54509a69 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:17:58.527883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9d384992-c2b1-47e6-bcae-b31e54509a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:17:58.591706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 582/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:17:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:17:58.858627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2588f-9217-462b-9058-5b6c293df1ed None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:58.913910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:58.958759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2588f-9217-462b-9058-5b6c293df1ed None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:58.992911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa0499-94b2-4cc7-893d-1dd5d5aeccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:58.993543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa0499-94b2-4cc7-893d-1dd5d5aeccfa None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:58.993823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa0499-94b2-4cc7-893d-1dd5d5aeccfa None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:58.994097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa0499-94b2-4cc7-893d-1dd5d5aeccfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:59.004483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa0499-94b2-4cc7-893d-1dd5d5aeccfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-1cfa0499-94b2-4cc7-893d-1dd5d5aeccfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:17:59.009728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:17:59.010874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:17:59.011799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 559/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:17:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:17:59.016945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d384992-c2b1-47e6-bcae-b31e54509a69 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:59.115051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d384992-c2b1-47e6-bcae-b31e54509a69 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:59.138027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:59.150170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d4e96-1f47-406d-a0bf-5792e9c640c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:59.150775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4e96-1f47-406d-a0bf-5792e9c640c2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:59.151048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4e96-1f47-406d-a0bf-5792e9c640c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:59.151315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4e96-1f47-406d-a0bf-5792e9c640c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:59.151974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d4e96-1f47-406d-a0bf-5792e9c640c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:59.152270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d4e96-1f47-406d-a0bf-5792e9c640c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:59.153660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d4e96-1f47-406d-a0bf-5792e9c640c2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:59.161689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 583/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:17:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:59.168477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa0499-94b2-4cc7-893d-1dd5d5aeccfa None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:59.264283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa0499-94b2-4cc7-893d-1dd5d5aeccfa None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:59.277948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:59.297777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06e585-f505-4947-8ea4-d1393b3b9916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:59.298390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06e585-f505-4947-8ea4-d1393b3b9916 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:59.298663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06e585-f505-4947-8ea4-d1393b3b9916 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:59.298931 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06e585-f505-4947-8ea4-d1393b3b9916 None admin] PATH_INFO: `/v3/users/e93e6dcb500f41a09747b5c22ca59715` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:59.311993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06e585-f505-4947-8ea4-d1393b3b9916 None admin] RBAC: Authorizing `identity:get_user(user_id=e93e6dcb500f41a09747b5c22ca59715)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:59.313690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06e585-f505-4947-8ea4-d1393b3b9916 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:59.315237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 560/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:17:59 2022] GET /identity/v3/users/e93e6dcb500f41a09747b5c22ca59715 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:59.321055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d4e96-1f47-406d-a0bf-5792e9c640c2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:59.417365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d4e96-1f47-406d-a0bf-5792e9c640c2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:59.448869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb17b3-fc97-49fd-87c0-95b398494991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:59.449464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb17b3-fc97-49fd-87c0-95b398494991 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:59.449748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb17b3-fc97-49fd-87c0-95b398494991 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:59.450018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb17b3-fc97-49fd-87c0-95b398494991 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:59.501236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:59.514055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb17b3-fc97-49fd-87c0-95b398494991 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:59.515490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb17b3-fc97-49fd-87c0-95b398494991 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:59.608879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 584/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:17:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 30 19:17:59.614773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06e585-f505-4947-8ea4-d1393b3b9916 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:59.709444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06e585-f505-4947-8ea4-d1393b3b9916 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:59.742224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f53b2c-c6b0-4b59-8b27-72d6f89ad1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:59.742828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f53b2c-c6b0-4b59-8b27-72d6f89ad1ba None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:59.743103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f53b2c-c6b0-4b59-8b27-72d6f89ad1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:59.743372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f53b2c-c6b0-4b59-8b27-72d6f89ad1ba None admin] PATH_INFO: `/v3/projects/16122c4c1b8e4995b419fa47a5d21b02` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:59.744488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f53b2c-c6b0-4b59-8b27-72d6f89ad1ba None admin] RBAC: Authorizing `identity:get_project(project_id=16122c4c1b8e4995b419fa47a5d21b02)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:59.745887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f53b2c-c6b0-4b59-8b27-72d6f89ad1ba None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:59.747458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 561/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:17:59 2022] GET /identity/v3/projects/16122c4c1b8e4995b419fa47a5d21b02 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:17:59.753328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb17b3-fc97-49fd-87c0-95b398494991 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:17:59.832482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:17:59.846656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb17b3-fc97-49fd-87c0-95b398494991 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:17:59.878859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009d3bf-116d-472c-a38a-bdb3db61b27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:17:59.879438 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009d3bf-116d-472c-a38a-bdb3db61b27e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:17:59.879707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009d3bf-116d-472c-a38a-bdb3db61b27e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:17:59.879969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009d3bf-116d-472c-a38a-bdb3db61b27e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:17:59.880697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009d3bf-116d-472c-a38a-bdb3db61b27e None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=16122c4c1b8e4995b419fa47a5d21b02, user.id=e93e6dcb500f41a09747b5c22ca59715) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:17:59.880995 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009d3bf-116d-472c-a38a-bdb3db61b27e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:17:59.882372 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009d3bf-116d-472c-a38a-bdb3db61b27e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:17:59.887396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 585/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:17:59 2022] GET /identity/v3/role_assignments?user.id=e93e6dcb500f41a09747b5c22ca59715&scope.project.id=16122c4c1b8e4995b419fa47a5d21b02&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:01.982325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc55c8-83bd-458a-8603-88c0bc030361 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:01.982857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc55c8-83bd-458a-8603-88c0bc030361 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:01.982857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc55c8-83bd-458a-8603-88c0bc030361 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:01.984177 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 562/1147] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:18:01.989378 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20241b-e268-4d55-aedc-7588cc67df15 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:01.989777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20241b-e268-4d55-aedc-7588cc67df15 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:01.989920 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20241b-e268-4d55-aedc-7588cc67df15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:02.009087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5a20241b-e268-4d55-aedc-7588cc67df15 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:18:02.010262 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5a20241b-e268-4d55-aedc-7588cc67df15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:18:02.073858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 586/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:18:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:18:02.337688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc55c8-83bd-458a-8603-88c0bc030361 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:02.379094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:02.436589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc55c8-83bd-458a-8603-88c0bc030361 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:02.469555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2039a5-241c-438b-bca1-bf088b95e982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:02.470171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2039a5-241c-438b-bca1-bf088b95e982 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:02.470446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2039a5-241c-438b-bca1-bf088b95e982 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:02.470715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2039a5-241c-438b-bca1-bf088b95e982 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:02.480462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2039a5-241c-438b-bca1-bf088b95e982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7a2039a5-241c-438b-bca1-bf088b95e982 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:18:02.485159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:18:02.486270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:18:02.487183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 563/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:18:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:18:02.492829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20241b-e268-4d55-aedc-7588cc67df15 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:02.570298 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:02.589305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20241b-e268-4d55-aedc-7588cc67df15 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:02.621357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396379f4-e9a6-4b42-adb7-50cd4e845881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:02.621954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396379f4-e9a6-4b42-adb7-50cd4e845881 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:02.622228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396379f4-e9a6-4b42-adb7-50cd4e845881 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:02.622495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396379f4-e9a6-4b42-adb7-50cd4e845881 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:02.623145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396379f4-e9a6-4b42-adb7-50cd4e845881 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:18:02.623441 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396379f4-e9a6-4b42-adb7-50cd4e845881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:02.624817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396379f4-e9a6-4b42-adb7-50cd4e845881 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:02.632937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 587/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:18:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:02.638779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2039a5-241c-438b-bca1-bf088b95e982 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:02.732701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:02.735849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2039a5-241c-438b-bca1-bf088b95e982 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:02.769314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6cf94-3355-407f-8e1c-8413e23d2ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:02.769924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6cf94-3355-407f-8e1c-8413e23d2ebd None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:02.770204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6cf94-3355-407f-8e1c-8413e23d2ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:02.770476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6cf94-3355-407f-8e1c-8413e23d2ebd None admin] PATH_INFO: `/v3/users/e93e6dcb500f41a09747b5c22ca59715` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:02.771642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6cf94-3355-407f-8e1c-8413e23d2ebd None admin] RBAC: Authorizing `identity:get_user(user_id=e93e6dcb500f41a09747b5c22ca59715)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:02.773095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6cf94-3355-407f-8e1c-8413e23d2ebd None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:02.774634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 564/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:18:02 2022] GET /identity/v3/users/e93e6dcb500f41a09747b5c22ca59715 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:02.780280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396379f4-e9a6-4b42-adb7-50cd4e845881 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:02.875441 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396379f4-e9a6-4b42-adb7-50cd4e845881 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:02.906168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cb490-0409-4993-be1b-d3aee2ff970d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:02.906768 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cb490-0409-4993-be1b-d3aee2ff970d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:02.907040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cb490-0409-4993-be1b-d3aee2ff970d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:02.907307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cb490-0409-4993-be1b-d3aee2ff970d None admin] PATH_INFO: `/v3/projects/16122c4c1b8e4995b419fa47a5d21b02` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:02.908401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94cb490-0409-4993-be1b-d3aee2ff970d None admin] RBAC: Authorizing `identity:get_project(project_id=16122c4c1b8e4995b419fa47a5d21b02)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:02.909807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94cb490-0409-4993-be1b-d3aee2ff970d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:02.911371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 588/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:18:02 2022] GET /identity/v3/projects/16122c4c1b8e4995b419fa47a5d21b02 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:02.917868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6cf94-3355-407f-8e1c-8413e23d2ebd None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:03.013625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6cf94-3355-407f-8e1c-8413e23d2ebd None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:03.066499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131ff2a-347d-4ed6-ade7-f781a2abab39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:03.067241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131ff2a-347d-4ed6-ade7-f781a2abab39 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:03.067524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131ff2a-347d-4ed6-ade7-f781a2abab39 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:03.067796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131ff2a-347d-4ed6-ade7-f781a2abab39 None admin] PATH_INFO: `/v3/projects/16122c4c1b8e4995b419fa47a5d21b02/users/e93e6dcb500f41a09747b5c22ca59715/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:03.073904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131ff2a-347d-4ed6-ade7-f781a2abab39 None admin] RBAC: Authorizing `identity:create_grant(project_id=16122c4c1b8e4995b419fa47a5d21b02, user_id=e93e6dcb500f41a09747b5c22ca59715, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:03.075405 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131ff2a-347d-4ed6-ade7-f781a2abab39 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:03.093257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 565/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:18:02 2022] PUT /identity/v3/projects/16122c4c1b8e4995b419fa47a5d21b02/users/e93e6dcb500f41a09747b5c22ca59715/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:18:05.198425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a06e7a-0a22-418a-bb59-95d43cbdaf9d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:05.199062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a06e7a-0a22-418a-bb59-95d43cbdaf9d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:05.199062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a06e7a-0a22-418a-bb59-95d43cbdaf9d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:05.200291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 589/1154] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:18:05.205586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575403b-e08b-4ee5-b2e8-8e76c0108110 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:05.206021 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575403b-e08b-4ee5-b2e8-8e76c0108110 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:05.206119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575403b-e08b-4ee5-b2e8-8e76c0108110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:05.223726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f575403b-e08b-4ee5-b2e8-8e76c0108110 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:18:05.225447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f575403b-e08b-4ee5-b2e8-8e76c0108110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:18:05.247606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:05.291295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 566/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:18:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:18:05.561579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a06e7a-0a22-418a-bb59-95d43cbdaf9d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:05.575277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:05.658535 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a06e7a-0a22-418a-bb59-95d43cbdaf9d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:05.689932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9393b34-de46-42e8-afbc-38ff2d6f7ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:05.690536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9393b34-de46-42e8-afbc-38ff2d6f7ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:05.690808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9393b34-de46-42e8-afbc-38ff2d6f7ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:05.691072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9393b34-de46-42e8-afbc-38ff2d6f7ec4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:05.701421 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9393b34-de46-42e8-afbc-38ff2d6f7ec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a9393b34-de46-42e8-afbc-38ff2d6f7ec4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:18:05.706070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:18:05.707261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:18:05.708454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 590/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:18:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:18:05.713708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575403b-e08b-4ee5-b2e8-8e76c0108110 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:05.807617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575403b-e08b-4ee5-b2e8-8e76c0108110 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:05.840174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9a892-0ffb-4db0-8dc0-29e0568fb2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:05.840730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9a892-0ffb-4db0-8dc0-29e0568fb2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:05.841002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9a892-0ffb-4db0-8dc0-29e0568fb2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:05.841265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9a892-0ffb-4db0-8dc0-29e0568fb2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:05.841908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9a892-0ffb-4db0-8dc0-29e0568fb2c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:18:05.842201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9a892-0ffb-4db0-8dc0-29e0568fb2c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:05.843544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9a892-0ffb-4db0-8dc0-29e0568fb2c3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:05.851142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 567/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:18:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:05.856931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9393b34-de46-42e8-afbc-38ff2d6f7ec4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:05.922671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:05.951481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9393b34-de46-42e8-afbc-38ff2d6f7ec4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:05.983201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1164d-9203-4709-9b8e-e2cf1b850ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:05.983802 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1164d-9203-4709-9b8e-e2cf1b850ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:05.984073 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1164d-9203-4709-9b8e-e2cf1b850ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:05.984338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1164d-9203-4709-9b8e-e2cf1b850ac7 None admin] PATH_INFO: `/v3/users/e93e6dcb500f41a09747b5c22ca59715` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:05.985480 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1164d-9203-4709-9b8e-e2cf1b850ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=e93e6dcb500f41a09747b5c22ca59715)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:05.986879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1164d-9203-4709-9b8e-e2cf1b850ac7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:05.988429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 591/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:18:05 2022] GET /identity/v3/users/e93e6dcb500f41a09747b5c22ca59715 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:05.994630 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9a892-0ffb-4db0-8dc0-29e0568fb2c3 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:06.015593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:06.091268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9a892-0ffb-4db0-8dc0-29e0568fb2c3 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:06.123146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d41ea-108b-423b-bcef-9880442ae46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:06.123730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d41ea-108b-423b-bcef-9880442ae46f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:06.124003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d41ea-108b-423b-bcef-9880442ae46f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:06.124270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d41ea-108b-423b-bcef-9880442ae46f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:06.188054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d41ea-108b-423b-bcef-9880442ae46f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:06.189502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d41ea-108b-423b-bcef-9880442ae46f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:06.209624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:06.284779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 568/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:18:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:18:06.291628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1164d-9203-4709-9b8e-e2cf1b850ac7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:06.386638 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1164d-9203-4709-9b8e-e2cf1b850ac7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:06.418966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4602c6-70b4-4bc8-84af-fce52980b361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:06.419585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4602c6-70b4-4bc8-84af-fce52980b361 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:06.419859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4602c6-70b4-4bc8-84af-fce52980b361 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:06.420126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4602c6-70b4-4bc8-84af-fce52980b361 None admin] PATH_INFO: `/v3/projects/16122c4c1b8e4995b419fa47a5d21b02` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:06.421238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4602c6-70b4-4bc8-84af-fce52980b361 None admin] RBAC: Authorizing `identity:get_project(project_id=16122c4c1b8e4995b419fa47a5d21b02)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:06.422646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4602c6-70b4-4bc8-84af-fce52980b361 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:06.424240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 592/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:18:06 2022] GET /identity/v3/projects/16122c4c1b8e4995b419fa47a5d21b02 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:06.430247 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d41ea-108b-423b-bcef-9880442ae46f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:06.525555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d41ea-108b-423b-bcef-9880442ae46f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:06.545340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:06.559040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd393a56-e55e-4dc8-9379-0c0685164955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:06.559633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd393a56-e55e-4dc8-9379-0c0685164955 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:06.559911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd393a56-e55e-4dc8-9379-0c0685164955 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:06.560182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd393a56-e55e-4dc8-9379-0c0685164955 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:06.560940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd393a56-e55e-4dc8-9379-0c0685164955 None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=16122c4c1b8e4995b419fa47a5d21b02, user.id=e93e6dcb500f41a09747b5c22ca59715) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:18:06.561246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd393a56-e55e-4dc8-9379-0c0685164955 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:06.562656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd393a56-e55e-4dc8-9379-0c0685164955 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:06.568800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 569/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:18:06 2022] GET /identity/v3/role_assignments?user.id=e93e6dcb500f41a09747b5c22ca59715&scope.project.id=16122c4c1b8e4995b419fa47a5d21b02&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:08.719330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654927ed-2fc7-44d9-bcaa-d719ec35b238 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:08.719853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654927ed-2fc7-44d9-bcaa-d719ec35b238 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:08.719853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654927ed-2fc7-44d9-bcaa-d719ec35b238 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:08.721215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 593/1162] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:18:08.726927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a84a04-1974-4e0d-8b67-6d2533cf073b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:08.727343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a84a04-1974-4e0d-8b67-6d2533cf073b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:08.727468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a84a04-1974-4e0d-8b67-6d2533cf073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:08.745645 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b0a84a04-1974-4e0d-8b67-6d2533cf073b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:18:08.746798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b0a84a04-1974-4e0d-8b67-6d2533cf073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:18:08.810384 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 570/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:18:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:18:09.075931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654927ed-2fc7-44d9-bcaa-d719ec35b238 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:09.085511 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:09.173230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654927ed-2fc7-44d9-bcaa-d719ec35b238 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:09.204259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bfb22-fcf0-45a6-86a5-49a78105a36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:09.204887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bfb22-fcf0-45a6-86a5-49a78105a36e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:09.205164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bfb22-fcf0-45a6-86a5-49a78105a36e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:09.205433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bfb22-fcf0-45a6-86a5-49a78105a36e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:09.268465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bfb22-fcf0-45a6-86a5-49a78105a36e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:09.269892 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bfb22-fcf0-45a6-86a5-49a78105a36e None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:09.278055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:09.365495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 594/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:18:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:18:09.371498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a84a04-1974-4e0d-8b67-6d2533cf073b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:09.467008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a84a04-1974-4e0d-8b67-6d2533cf073b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:09.480910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:09.500450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31d8fa-5d96-4f07-8762-acd2ce39ca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:09.501066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31d8fa-5d96-4f07-8762-acd2ce39ca11 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:09.501343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31d8fa-5d96-4f07-8762-acd2ce39ca11 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:09.501609 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31d8fa-5d96-4f07-8762-acd2ce39ca11 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:09.508584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31d8fa-5d96-4f07-8762-acd2ce39ca11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:09.510005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31d8fa-5d96-4f07-8762-acd2ce39ca11 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-2e31d8fa-5d96-4f07-8762-acd2ce39ca11 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:18:09.521051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:18:09.522815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:18:09.523739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 571/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 30 19:18:09 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:18:09.528744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bfb22-fcf0-45a6-86a5-49a78105a36e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:09.621966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bfb22-fcf0-45a6-86a5-49a78105a36e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:09.628666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:09.654660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aeeeb-122f-443d-9cfe-31296718a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:09.655236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aeeeb-122f-443d-9cfe-31296718a380 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:09.655508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aeeeb-122f-443d-9cfe-31296718a380 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:09.655772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aeeeb-122f-443d-9cfe-31296718a380 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:09.656413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aeeeb-122f-443d-9cfe-31296718a380 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:18:09.656710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aeeeb-122f-443d-9cfe-31296718a380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:09.658049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aeeeb-122f-443d-9cfe-31296718a380 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:09.665015 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 595/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Sep 30 19:18:09 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:11.760903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d279ef4-439b-4af0-a7d8-70d38625e1da None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:11.761434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d279ef4-439b-4af0-a7d8-70d38625e1da None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:11.761434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d279ef4-439b-4af0-a7d8-70d38625e1da None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:11.762758 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 572/1167] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:18:11.767843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d9005-bcca-484e-844c-cb61e4916327 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:11.768224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d9005-bcca-484e-844c-cb61e4916327 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:11.768388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d9005-bcca-484e-844c-cb61e4916327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:11.785759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-340d9005-bcca-484e-844c-cb61e4916327 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:18:11.787439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-340d9005-bcca-484e-844c-cb61e4916327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:18:11.849023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 596/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:18:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:18:12.112294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d279ef4-439b-4af0-a7d8-70d38625e1da None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:12.212787 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d279ef4-439b-4af0-a7d8-70d38625e1da None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:12.247057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5bbfb-f5b5-4cb9-8ed9-d1cd4707f8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:12.247648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5bbfb-f5b5-4cb9-8ed9-d1cd4707f8eb None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:12.247919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5bbfb-f5b5-4cb9-8ed9-d1cd4707f8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:12.248184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5bbfb-f5b5-4cb9-8ed9-d1cd4707f8eb None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:12.248871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5bbfb-f5b5-4cb9-8ed9-d1cd4707f8eb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:12.250215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5bbfb-f5b5-4cb9-8ed9-d1cd4707f8eb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:12.275740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 573/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Fri Sep 30 19:18:12 2022] POST /identity/v3/domains => generated 267 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:18:14.429971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801c70-8339-4a27-b933-916ff28c779b None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:14.430413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801c70-8339-4a27-b933-916ff28c779b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:14.430507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801c70-8339-4a27-b933-916ff28c779b None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:14.431849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 597/1170] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:18:14.437092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb489a58-122a-4469-a2c0-9d73e78d066b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:14.437509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb489a58-122a-4469-a2c0-9d73e78d066b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:14.437635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb489a58-122a-4469-a2c0-9d73e78d066b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:14.455802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cb489a58-122a-4469-a2c0-9d73e78d066b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:18:14.456992 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cb489a58-122a-4469-a2c0-9d73e78d066b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:18:14.482185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:14.522140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 574/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:18:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:18:14.793073 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d801c70-8339-4a27-b933-916ff28c779b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:14.885858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:14.890062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d801c70-8339-4a27-b933-916ff28c779b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:14.922289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3150299-9bad-4b45-95b8-bf637af651ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:14.922900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150299-9bad-4b45-95b8-bf637af651ed None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:14.923176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150299-9bad-4b45-95b8-bf637af651ed None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:14.923447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3150299-9bad-4b45-95b8-bf637af651ed None admin] PATH_INFO: `/v3/domains/d3c204975cd34976a1be38e7973ec788` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:14.933307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3150299-9bad-4b45-95b8-bf637af651ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=d3c204975cd34976a1be38e7973ec788)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:14.934722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3150299-9bad-4b45-95b8-bf637af651ed None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:14.936258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 598/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Fri Sep 30 19:18:14 2022] GET /identity/v3/domains/d3c204975cd34976a1be38e7973ec788 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:14.941932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb489a58-122a-4469-a2c0-9d73e78d066b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:15.036781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb489a58-122a-4469-a2c0-9d73e78d066b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:15.070278 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9500fe4-7c61-41fa-b74b-8e6ecebde4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:15.070811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9500fe4-7c61-41fa-b74b-8e6ecebde4a9 None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:15.071085 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9500fe4-7c61-41fa-b74b-8e6ecebde4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:15.071352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9500fe4-7c61-41fa-b74b-8e6ecebde4a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:15.071986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9500fe4-7c61-41fa-b74b-8e6ecebde4a9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:15.073441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9500fe4-7c61-41fa-b74b-8e6ecebde4a9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:15.097549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 575/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:18:14 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:18:17.251267 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b74a8-5725-4ed2-a5d2-511b4133c9ba None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:17.251831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b74a8-5725-4ed2-a5d2-511b4133c9ba None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:17.251831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b74a8-5725-4ed2-a5d2-511b4133c9ba None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:17.253167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 599/1174] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:18:17.258344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18ffc5-3feb-4daa-b145-60073555b34e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:17.258748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18ffc5-3feb-4daa-b145-60073555b34e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:17.258887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18ffc5-3feb-4daa-b145-60073555b34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:17.276941 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bc18ffc5-3feb-4daa-b145-60073555b34e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:18:17.278087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bc18ffc5-3feb-4daa-b145-60073555b34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:18:17.289135 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:17.342844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 576/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:18:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:18:17.608301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b74a8-5725-4ed2-a5d2-511b4133c9ba None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:17.705167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b74a8-5725-4ed2-a5d2-511b4133c9ba None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:17.736785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56931967-23e7-4cc0-93ff-70941960a8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:17.737392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56931967-23e7-4cc0-93ff-70941960a8cc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:17.737670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56931967-23e7-4cc0-93ff-70941960a8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:17.737938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56931967-23e7-4cc0-93ff-70941960a8cc None admin] PATH_INFO: `/v3/domains/d3c204975cd34976a1be38e7973ec788` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:17.739034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56931967-23e7-4cc0-93ff-70941960a8cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=d3c204975cd34976a1be38e7973ec788)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:17.740504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56931967-23e7-4cc0-93ff-70941960a8cc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:17.742013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 600/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Fri Sep 30 19:18:17 2022] GET /identity/v3/domains/d3c204975cd34976a1be38e7973ec788 => generated 267 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:17.747517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18ffc5-3feb-4daa-b145-60073555b34e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:17.841622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18ffc5-3feb-4daa-b145-60073555b34e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:17.874382 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ab4ce-3a9b-427c-a2ce-fd6db671030d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:17.874911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ab4ce-3a9b-427c-a2ce-fd6db671030d None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:17.875191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ab4ce-3a9b-427c-a2ce-fd6db671030d None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:17.875458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ab4ce-3a9b-427c-a2ce-fd6db671030d None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:17.876106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ab4ce-3a9b-427c-a2ce-fd6db671030d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:17.877582 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ab4ce-3a9b-427c-a2ce-fd6db671030d None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:17.899123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 577/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:18:17 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:18:20.060392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd6fee-dead-48c3-893a-7b54c6b96d59 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:20.060923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd6fee-dead-48c3-893a-7b54c6b96d59 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:20.060923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd6fee-dead-48c3-893a-7b54c6b96d59 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:20.062267 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 601/1178] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:18:20.067395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e271fc4-69a6-40e4-a5b3-26053b6f16aa None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:20.067831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e271fc4-69a6-40e4-a5b3-26053b6f16aa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:20.068580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e271fc4-69a6-40e4-a5b3-26053b6f16aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:20.074779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:20.088279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8e271fc4-69a6-40e4-a5b3-26053b6f16aa None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:18:20.089469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8e271fc4-69a6-40e4-a5b3-26053b6f16aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:18:20.151871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 578/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:18:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:18:20.422356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd6fee-dead-48c3-893a-7b54c6b96d59 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:20.449566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:20.519062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd6fee-dead-48c3-893a-7b54c6b96d59 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:20.551608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5afdd0-dfe3-4224-b7fc-16f228a7dc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:20.552214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5afdd0-dfe3-4224-b7fc-16f228a7dc4e None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:20.552538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5afdd0-dfe3-4224-b7fc-16f228a7dc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:20.552807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5afdd0-dfe3-4224-b7fc-16f228a7dc4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:20.562722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5afdd0-dfe3-4224-b7fc-16f228a7dc4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-dd5afdd0-dfe3-4224-b7fc-16f228a7dc4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:18:20.567410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:18:20.568579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:18:20.569655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 602/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:18:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:18:20.574409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e271fc4-69a6-40e4-a5b3-26053b6f16aa None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:20.673856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e271fc4-69a6-40e4-a5b3-26053b6f16aa None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:20.701518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:20.706544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0d1f8-e362-49eb-87a2-147d43df99cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:20.707133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d1f8-e362-49eb-87a2-147d43df99cb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:20.707406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d1f8-e362-49eb-87a2-147d43df99cb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:20.707673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d1f8-e362-49eb-87a2-147d43df99cb None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:20.708321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0d1f8-e362-49eb-87a2-147d43df99cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:18:20.709238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0d1f8-e362-49eb-87a2-147d43df99cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:20.710584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0d1f8-e362-49eb-87a2-147d43df99cb None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:20.718282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 579/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:18:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:20.724129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5afdd0-dfe3-4224-b7fc-16f228a7dc4e None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:20.807556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:20.819742 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5afdd0-dfe3-4224-b7fc-16f228a7dc4e None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:20.851478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad3aa5-1695-4a0e-8c80-030b56f4a01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:20.852082 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad3aa5-1695-4a0e-8c80-030b56f4a01c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:20.852355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad3aa5-1695-4a0e-8c80-030b56f4a01c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:20.852649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad3aa5-1695-4a0e-8c80-030b56f4a01c None admin] PATH_INFO: `/v3/users/d64e490747554b2aad1567e485cefabe` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:20.864867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad3aa5-1695-4a0e-8c80-030b56f4a01c None admin] RBAC: Authorizing `identity:get_user(user_id=d64e490747554b2aad1567e485cefabe)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:20.866292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad3aa5-1695-4a0e-8c80-030b56f4a01c None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:20.867820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 603/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:18:20 2022] GET /identity/v3/users/d64e490747554b2aad1567e485cefabe => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:20.873495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0d1f8-e362-49eb-87a2-147d43df99cb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:20.968865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0d1f8-e362-49eb-87a2-147d43df99cb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:21.000516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be636423-e7c2-41f8-94a0-db4189ae61d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:21.001091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be636423-e7c2-41f8-94a0-db4189ae61d4 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:21.001362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be636423-e7c2-41f8-94a0-db4189ae61d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:21.001623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be636423-e7c2-41f8-94a0-db4189ae61d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:21.058742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:21.066366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be636423-e7c2-41f8-94a0-db4189ae61d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:21.067806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be636423-e7c2-41f8-94a0-db4189ae61d4 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:21.163913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 580/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:18:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:18:21.169796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad3aa5-1695-4a0e-8c80-030b56f4a01c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:21.264260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad3aa5-1695-4a0e-8c80-030b56f4a01c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:21.296585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7556e2d-c448-40a1-a1c0-a8402007a392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:21.297194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7556e2d-c448-40a1-a1c0-a8402007a392 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:21.297467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7556e2d-c448-40a1-a1c0-a8402007a392 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:21.297734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7556e2d-c448-40a1-a1c0-a8402007a392 None admin] PATH_INFO: `/v3/projects/ed5ee1dc3959404ebf7c1f53d3e66aa2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:21.298818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7556e2d-c448-40a1-a1c0-a8402007a392 None admin] RBAC: Authorizing `identity:get_project(project_id=ed5ee1dc3959404ebf7c1f53d3e66aa2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:21.300222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7556e2d-c448-40a1-a1c0-a8402007a392 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:21.301854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 604/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:18:21 2022] GET /identity/v3/projects/ed5ee1dc3959404ebf7c1f53d3e66aa2 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:21.308259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be636423-e7c2-41f8-94a0-db4189ae61d4 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:21.397235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:21.404415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be636423-e7c2-41f8-94a0-db4189ae61d4 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:21.437763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b827ab-e9f2-4ae0-af17-c033e3028d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:21.438345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b827ab-e9f2-4ae0-af17-c033e3028d0a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:21.438618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b827ab-e9f2-4ae0-af17-c033e3028d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:21.438881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b827ab-e9f2-4ae0-af17-c033e3028d0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:21.439583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b827ab-e9f2-4ae0-af17-c033e3028d0a None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=ed5ee1dc3959404ebf7c1f53d3e66aa2, user.id=d64e490747554b2aad1567e485cefabe) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:18:21.439882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b827ab-e9f2-4ae0-af17-c033e3028d0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:21.441321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b827ab-e9f2-4ae0-af17-c033e3028d0a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:21.446586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 581/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:18:21 2022] GET /identity/v3/role_assignments?user.id=d64e490747554b2aad1567e485cefabe&scope.project.id=ed5ee1dc3959404ebf7c1f53d3e66aa2&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:23.542248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d79cb-c7c6-4d50-9426-076a3ad20dee None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:23.542797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d79cb-c7c6-4d50-9426-076a3ad20dee None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:23.542797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d79cb-c7c6-4d50-9426-076a3ad20dee None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:23.544090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 605/1186] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:18:23.549169 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbdf1a-c1a2-428f-a715-8a4f6fc9dca2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:23.549542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbdf1a-c1a2-428f-a715-8a4f6fc9dca2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:23.549712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbdf1a-c1a2-428f-a715-8a4f6fc9dca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:23.568662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ecbbdf1a-c1a2-428f-a715-8a4f6fc9dca2 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:18:23.569817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ecbbdf1a-c1a2-428f-a715-8a4f6fc9dca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:18:23.632798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 582/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:18:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:18:23.903427 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d79cb-c7c6-4d50-9426-076a3ad20dee None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:23.919384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:24.000248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d79cb-c7c6-4d50-9426-076a3ad20dee None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:24.031761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c44e60-0ad6-4e4d-9876-1e62481e39d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:24.032378 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c44e60-0ad6-4e4d-9876-1e62481e39d4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:24.032661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c44e60-0ad6-4e4d-9876-1e62481e39d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:24.032924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c44e60-0ad6-4e4d-9876-1e62481e39d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:24.042822 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c44e60-0ad6-4e4d-9876-1e62481e39d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-47c44e60-0ad6-4e4d-9876-1e62481e39d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:18:24.047531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:18:24.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:18:24.049945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 606/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:18:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:18:24.055597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbbdf1a-c1a2-428f-a715-8a4f6fc9dca2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:24.139643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:24.152174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbbdf1a-c1a2-428f-a715-8a4f6fc9dca2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:24.185390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6b468-f544-4870-afd1-2c64781c814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:24.185983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6b468-f544-4870-afd1-2c64781c814a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:24.186259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6b468-f544-4870-afd1-2c64781c814a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:24.186529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6b468-f544-4870-afd1-2c64781c814a None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:24.187180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6b468-f544-4870-afd1-2c64781c814a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:18:24.187479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6b468-f544-4870-afd1-2c64781c814a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:24.188870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6b468-f544-4870-afd1-2c64781c814a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:24.196679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 583/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:18:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:24.202484 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c44e60-0ad6-4e4d-9876-1e62481e39d4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:24.273716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:24.298011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c44e60-0ad6-4e4d-9876-1e62481e39d4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:24.335951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a848f-9390-48a8-a528-5c981d498192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:24.336589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a848f-9390-48a8-a528-5c981d498192 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:24.336873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a848f-9390-48a8-a528-5c981d498192 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:24.337127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a848f-9390-48a8-a528-5c981d498192 None admin] PATH_INFO: `/v3/users/d64e490747554b2aad1567e485cefabe` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:24.338210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a848f-9390-48a8-a528-5c981d498192 None admin] RBAC: Authorizing `identity:get_user(user_id=d64e490747554b2aad1567e485cefabe)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:24.339612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a848f-9390-48a8-a528-5c981d498192 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:24.341162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 607/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:18:24 2022] GET /identity/v3/users/d64e490747554b2aad1567e485cefabe => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:24.346928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6b468-f544-4870-afd1-2c64781c814a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:24.448686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6b468-f544-4870-afd1-2c64781c814a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:24.480546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148990e5-e7a1-446f-84b9-7235134f47b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:24.481156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148990e5-e7a1-446f-84b9-7235134f47b3 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:24.481428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148990e5-e7a1-446f-84b9-7235134f47b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:24.481695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148990e5-e7a1-446f-84b9-7235134f47b3 None admin] PATH_INFO: `/v3/projects/ed5ee1dc3959404ebf7c1f53d3e66aa2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:24.482785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148990e5-e7a1-446f-84b9-7235134f47b3 None admin] RBAC: Authorizing `identity:get_project(project_id=ed5ee1dc3959404ebf7c1f53d3e66aa2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:24.484194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148990e5-e7a1-446f-84b9-7235134f47b3 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:24.485825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 584/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:18:24 2022] GET /identity/v3/projects/ed5ee1dc3959404ebf7c1f53d3e66aa2 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:24.491946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a848f-9390-48a8-a528-5c981d498192 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:24.586585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a848f-9390-48a8-a528-5c981d498192 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:24.618016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68041f-e848-458d-87c2-983a6484f494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:24.618653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68041f-e848-458d-87c2-983a6484f494 None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:24.618932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68041f-e848-458d-87c2-983a6484f494 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:24.619204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68041f-e848-458d-87c2-983a6484f494 None admin] PATH_INFO: `/v3/projects/ed5ee1dc3959404ebf7c1f53d3e66aa2/users/d64e490747554b2aad1567e485cefabe/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:24.621097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68041f-e848-458d-87c2-983a6484f494 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed5ee1dc3959404ebf7c1f53d3e66aa2, user_id=d64e490747554b2aad1567e485cefabe, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:24.622501 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68041f-e848-458d-87c2-983a6484f494 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:24.635001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 608/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:18:24 2022] PUT /identity/v3/projects/ed5ee1dc3959404ebf7c1f53d3e66aa2/users/d64e490747554b2aad1567e485cefabe/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:18:26.745487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954b7cf-9b45-45ae-8d5f-5829ca63205f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:26.746005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954b7cf-9b45-45ae-8d5f-5829ca63205f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:26.746005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954b7cf-9b45-45ae-8d5f-5829ca63205f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:26.747356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 585/1193] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:18:26.752669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c84405-e83d-4bb0-bc11-85e6da773c95 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:26.753097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c84405-e83d-4bb0-bc11-85e6da773c95 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:26.753248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c84405-e83d-4bb0-bc11-85e6da773c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:26.759634 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:26.773188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-83c84405-e83d-4bb0-bc11-85e6da773c95 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:18:26.774387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-83c84405-e83d-4bb0-bc11-85e6da773c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:18:26.844788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 609/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:18:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:18:27.114270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954b7cf-9b45-45ae-8d5f-5829ca63205f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:27.134838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:27.214645 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954b7cf-9b45-45ae-8d5f-5829ca63205f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:27.246666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac5776-c998-4b3f-a54d-2c04f396a045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:27.247277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac5776-c998-4b3f-a54d-2c04f396a045 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:27.247555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac5776-c998-4b3f-a54d-2c04f396a045 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:27.247824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac5776-c998-4b3f-a54d-2c04f396a045 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:27.258086 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac5776-c998-4b3f-a54d-2c04f396a045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bdac5776-c998-4b3f-a54d-2c04f396a045 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:18:27.262815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:18:27.263950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:18:27.265066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 586/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:18:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:18:27.269932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c84405-e83d-4bb0-bc11-85e6da773c95 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:27.363329 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c84405-e83d-4bb0-bc11-85e6da773c95 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:27.390902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:27.396032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7dd9-6884-405c-863b-a6699635545f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:27.396673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7dd9-6884-405c-863b-a6699635545f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:27.396950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7dd9-6884-405c-863b-a6699635545f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:27.397219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7dd9-6884-405c-863b-a6699635545f None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:27.397879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a7dd9-6884-405c-863b-a6699635545f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:18:27.398178 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a7dd9-6884-405c-863b-a6699635545f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:27.399535 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a7dd9-6884-405c-863b-a6699635545f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:27.407552 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 610/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:18:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:27.414297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac5776-c998-4b3f-a54d-2c04f396a045 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:27.489937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:27.511436 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac5776-c998-4b3f-a54d-2c04f396a045 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:27.544184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb3759-7b93-4b76-b61a-7023a22f8ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:27.544822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb3759-7b93-4b76-b61a-7023a22f8ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:27.545098 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb3759-7b93-4b76-b61a-7023a22f8ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:27.545366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb3759-7b93-4b76-b61a-7023a22f8ee1 None admin] PATH_INFO: `/v3/users/d64e490747554b2aad1567e485cefabe` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:27.546458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb3759-7b93-4b76-b61a-7023a22f8ee1 None admin] RBAC: Authorizing `identity:get_user(user_id=d64e490747554b2aad1567e485cefabe)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:27.547866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb3759-7b93-4b76-b61a-7023a22f8ee1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:27.549434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 587/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Sep 30 19:18:27 2022] GET /identity/v3/users/d64e490747554b2aad1567e485cefabe => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:27.555131 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7dd9-6884-405c-863b-a6699635545f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:27.649433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7dd9-6884-405c-863b-a6699635545f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:27.680825 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6537b6-7bd7-45ca-b1f8-5ca3ba0fdace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:27.681412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6537b6-7bd7-45ca-b1f8-5ca3ba0fdace None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:27.681685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6537b6-7bd7-45ca-b1f8-5ca3ba0fdace None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:27.681951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6537b6-7bd7-45ca-b1f8-5ca3ba0fdace None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:27.737888 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:27.745300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6537b6-7bd7-45ca-b1f8-5ca3ba0fdace None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:27.746709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6537b6-7bd7-45ca-b1f8-5ca3ba0fdace None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:27.838211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 611/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:18:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:18:27.843983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb3759-7b93-4b76-b61a-7023a22f8ee1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:27.940610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb3759-7b93-4b76-b61a-7023a22f8ee1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:27.972587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e278c-0873-4d81-af65-c8801414e57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:27.973197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e278c-0873-4d81-af65-c8801414e57b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:27.973472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e278c-0873-4d81-af65-c8801414e57b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:27.973740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e278c-0873-4d81-af65-c8801414e57b None admin] PATH_INFO: `/v3/projects/ed5ee1dc3959404ebf7c1f53d3e66aa2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:27.975428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e278c-0873-4d81-af65-c8801414e57b None admin] RBAC: Authorizing `identity:get_project(project_id=ed5ee1dc3959404ebf7c1f53d3e66aa2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:27.976877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e278c-0873-4d81-af65-c8801414e57b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:27.978449 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 588/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Sep 30 19:18:27 2022] GET /identity/v3/projects/ed5ee1dc3959404ebf7c1f53d3e66aa2 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:27.984277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6537b6-7bd7-45ca-b1f8-5ca3ba0fdace None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:18:28.071085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:28.077721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6537b6-7bd7-45ca-b1f8-5ca3ba0fdace None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:18:28.110632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9484177-9ce7-4598-822f-b70706dd07e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:18:28.111220 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484177-9ce7-4598-822f-b70706dd07e0 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:18:28.111495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484177-9ce7-4598-822f-b70706dd07e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:18:28.111763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484177-9ce7-4598-822f-b70706dd07e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:18:28.112506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9484177-9ce7-4598-822f-b70706dd07e0 None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=ed5ee1dc3959404ebf7c1f53d3e66aa2, user.id=d64e490747554b2aad1567e485cefabe) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:18:28.112808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9484177-9ce7-4598-822f-b70706dd07e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:18:28.114215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9484177-9ce7-4598-822f-b70706dd07e0 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:18:28.119680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 612/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:18:27 2022] GET /identity/v3/role_assignments?user.id=d64e490747554b2aad1567e485cefabe&scope.project.id=ed5ee1dc3959404ebf7c1f53d3e66aa2&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:18:37.968629 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:18:38.117191 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=21 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:23.362828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3300-c3a3-402e-8f03-67f7a881534d None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:23.362828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3300-c3a3-402e-8f03-67f7a881534d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:23.362828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3300-c3a3-402e-8f03-67f7a881534d None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:23.362828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 589/1201] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:19:23.372403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1a37-97fa-4f73-817d-9c2eadba7182 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:23.372403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1a37-97fa-4f73-817d-9c2eadba7182 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:23.372403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1a37-97fa-4f73-817d-9c2eadba7182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:23.394260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ea4b1a37-97fa-4f73-817d-9c2eadba7182 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:19:23.395511 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ea4b1a37-97fa-4f73-817d-9c2eadba7182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:19:23.485293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 613/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:19:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:19:23.761613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3300-c3a3-402e-8f03-67f7a881534d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:23.886122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3300-c3a3-402e-8f03-67f7a881534d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:23.923673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e1e0-d9d6-4cc6-8c2d-b2c068ee6a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:23.924307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e1e0-d9d6-4cc6-8c2d-b2c068ee6a78 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:23.924612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e1e0-d9d6-4cc6-8c2d-b2c068ee6a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:23.924883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e1e0-d9d6-4cc6-8c2d-b2c068ee6a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:23.943890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640e1e0-d9d6-4cc6-8c2d-b2c068ee6a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:23.945405 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640e1e0-d9d6-4cc6-8c2d-b2c068ee6a78 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f640e1e0-d9d6-4cc6-8c2d-b2c068ee6a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:23.960185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:23.964592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:23.965667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:23.965667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:23.965667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:23.965667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 590/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:23.970113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b1a37-97fa-4f73-817d-9c2eadba7182 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:24.105653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b1a37-97fa-4f73-817d-9c2eadba7182 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:24.142339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:24.150688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd6625-d3ee-4c7e-8330-05c852a82e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:24.151394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd6625-d3ee-4c7e-8330-05c852a82e10 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:24.151682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd6625-d3ee-4c7e-8330-05c852a82e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:24.151971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd6625-d3ee-4c7e-8330-05c852a82e10 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:24.152644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd6625-d3ee-4c7e-8330-05c852a82e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:24.152986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd6625-d3ee-4c7e-8330-05c852a82e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:24.154392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd6625-d3ee-4c7e-8330-05c852a82e10 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:24.174689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e1e0-d9d6-4cc6-8c2d-b2c068ee6a78 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:24.174877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 614/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:24.248083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:24.308572 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e1e0-d9d6-4cc6-8c2d-b2c068ee6a78 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:24.355852 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d55e85-bc2c-446b-a28f-0cfc144d2d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:24.356463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d55e85-bc2c-446b-a28f-0cfc144d2d3c None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:24.356756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d55e85-bc2c-446b-a28f-0cfc144d2d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:24.357050 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d55e85-bc2c-446b-a28f-0cfc144d2d3c None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:24.357754 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d55e85-bc2c-446b-a28f-0cfc144d2d3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:24.359293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d55e85-bc2c-446b-a28f-0cfc144d2d3c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:24.391823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 591/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Sep 30 19:19:24 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:19:26.783866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78031442-923b-4063-8775-0eea16f23962 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:26.784424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78031442-923b-4063-8775-0eea16f23962 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:26.784424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78031442-923b-4063-8775-0eea16f23962 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:26.785803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 615/1206] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:19:26.793145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cf169-dbe6-404f-8ea6-abc74315dd05 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:26.793676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cf169-dbe6-404f-8ea6-abc74315dd05 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:26.793676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cf169-dbe6-404f-8ea6-abc74315dd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:26.814207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ef3cf169-dbe6-404f-8ea6-abc74315dd05 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:19:26.815469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ef3cf169-dbe6-404f-8ea6-abc74315dd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:19:26.890732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 592/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:19:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:19:27.220445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78031442-923b-4063-8775-0eea16f23962 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:27.329527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78031442-923b-4063-8775-0eea16f23962 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:27.367345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f890713-1a0e-4aca-9c67-98c9b4cd6206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:27.368619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f890713-1a0e-4aca-9c67-98c9b4cd6206 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:27.368981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f890713-1a0e-4aca-9c67-98c9b4cd6206 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:27.369272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f890713-1a0e-4aca-9c67-98c9b4cd6206 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:27.382532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f890713-1a0e-4aca-9c67-98c9b4cd6206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-4f890713-1a0e-4aca-9c67-98c9b4cd6206 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:19:27.387815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:27.388858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:27.390136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 616/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:19:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:19:27.396960 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cf169-dbe6-404f-8ea6-abc74315dd05 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:27.443998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:27.513876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cf169-dbe6-404f-8ea6-abc74315dd05 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:27.549888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684a191-f7e3-4730-8625-ace4747fa9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:27.550490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a191-f7e3-4730-8625-ace4747fa9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:27.550826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a191-f7e3-4730-8625-ace4747fa9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:27.551575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a191-f7e3-4730-8625-ace4747fa9d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:27.552262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684a191-f7e3-4730-8625-ace4747fa9d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:27.552638 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684a191-f7e3-4730-8625-ace4747fa9d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:27.553954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684a191-f7e3-4730-8625-ace4747fa9d9 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:27.562697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 593/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:19:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:27.568634 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f890713-1a0e-4aca-9c67-98c9b4cd6206 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:27.625512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:27.680157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f890713-1a0e-4aca-9c67-98c9b4cd6206 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:27.716212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dec79-ce7c-4000-9d1b-56caea2c283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:27.717349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dec79-ce7c-4000-9d1b-56caea2c283d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:27.717847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dec79-ce7c-4000-9d1b-56caea2c283d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:27.718370 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dec79-ce7c-4000-9d1b-56caea2c283d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:27.732943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66dec79-ce7c-4000-9d1b-56caea2c283d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:27.734636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66dec79-ce7c-4000-9d1b-56caea2c283d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d66dec79-ce7c-4000-9d1b-56caea2c283d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:27.754590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:27.755871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:27.757060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:27.757060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:27.757060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:27.758513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 617/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:19:27.763243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684a191-f7e3-4730-8625-ace4747fa9d9 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:27.881076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:27.887123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684a191-f7e3-4730-8625-ace4747fa9d9 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:27.927043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0df8a-6ac6-46d7-aa4a-fb05c7869000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:27.927701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0df8a-6ac6-46d7-aa4a-fb05c7869000 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:27.927983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0df8a-6ac6-46d7-aa4a-fb05c7869000 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:27.928251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0df8a-6ac6-46d7-aa4a-fb05c7869000 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:27.929055 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0df8a-6ac6-46d7-aa4a-fb05c7869000 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:27.929997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0df8a-6ac6-46d7-aa4a-fb05c7869000 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:27.930753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0df8a-6ac6-46d7-aa4a-fb05c7869000 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:27.943275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 594/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:27.950086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dec79-ce7c-4000-9d1b-56caea2c283d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:28.061080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dec79-ce7c-4000-9d1b-56caea2c283d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:28.062600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:28.103712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78369f2-7200-4fa5-8120-793f34570e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:28.104410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78369f2-7200-4fa5-8120-793f34570e9d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:28.104745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78369f2-7200-4fa5-8120-793f34570e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:28.104981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78369f2-7200-4fa5-8120-793f34570e9d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:28.114728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78369f2-7200-4fa5-8120-793f34570e9d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:28.116202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78369f2-7200-4fa5-8120-793f34570e9d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a78369f2-7200-4fa5-8120-793f34570e9d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:19:28.128642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:28.129965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:28.131214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:28.131214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:28.131214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:19:28.131214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:19:28.131214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:19:28.131214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:19:28.131214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:19:28.131214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:19:28.131214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:28.131214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:28.131214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 618/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:28.136628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0df8a-6ac6-46d7-aa4a-fb05c7869000 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:28.259832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0df8a-6ac6-46d7-aa4a-fb05c7869000 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:28.296232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba398885-da47-45e0-a79d-ee15391d751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:28.297227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba398885-da47-45e0-a79d-ee15391d751b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:28.297663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba398885-da47-45e0-a79d-ee15391d751b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:28.298167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba398885-da47-45e0-a79d-ee15391d751b None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:28.299148 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba398885-da47-45e0-a79d-ee15391d751b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:28.299607 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba398885-da47-45e0-a79d-ee15391d751b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:28.302016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba398885-da47-45e0-a79d-ee15391d751b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:28.323099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 595/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 30 19:19:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:28.328706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78369f2-7200-4fa5-8120-793f34570e9d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:28.439657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78369f2-7200-4fa5-8120-793f34570e9d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:28.476725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400be041-9517-44a5-8349-64a0451deca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:28.477676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400be041-9517-44a5-8349-64a0451deca9 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:28.478106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400be041-9517-44a5-8349-64a0451deca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:28.478539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400be041-9517-44a5-8349-64a0451deca9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:28.511843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:28.554465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400be041-9517-44a5-8349-64a0451deca9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:28.555964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400be041-9517-44a5-8349-64a0451deca9 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:28.687221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 619/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:19:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 30 19:19:28.694242 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba398885-da47-45e0-a79d-ee15391d751b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:28.713973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:28.801457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba398885-da47-45e0-a79d-ee15391d751b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:28.836823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a86149-b456-49a0-9e24-77b4951b7a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:28.837463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a86149-b456-49a0-9e24-77b4951b7a2a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:28.837740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a86149-b456-49a0-9e24-77b4951b7a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:28.838010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a86149-b456-49a0-9e24-77b4951b7a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:28.848906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a86149-b456-49a0-9e24-77b4951b7a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:28.850348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a86149-b456-49a0-9e24-77b4951b7a2a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-74a86149-b456-49a0-9e24-77b4951b7a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:28.866146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:28.867345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:28.868495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:28.868495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:28.868495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:28.872401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 596/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:28.873177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400be041-9517-44a5-8349-64a0451deca9 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:28.940416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:28.981385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400be041-9517-44a5-8349-64a0451deca9 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:29.015808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e43010-c028-4e37-b81f-42a934db01b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:29.016443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e43010-c028-4e37-b81f-42a934db01b6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:29.016717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e43010-c028-4e37-b81f-42a934db01b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:29.016983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e43010-c028-4e37-b81f-42a934db01b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:29.017622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e43010-c028-4e37-b81f-42a934db01b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:29.017918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e43010-c028-4e37-b81f-42a934db01b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:29.019278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e43010-c028-4e37-b81f-42a934db01b6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:29.032254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 620/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:19:29.039023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a86149-b456-49a0-9e24-77b4951b7a2a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:29.114443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:29.149359 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a86149-b456-49a0-9e24-77b4951b7a2a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:29.187655 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0ff25-292d-4433-a928-bf36b4ecab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:29.188570 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0ff25-292d-4433-a928-bf36b4ecab0f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:29.188948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0ff25-292d-4433-a928-bf36b4ecab0f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:29.189332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0ff25-292d-4433-a928-bf36b4ecab0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:29.197037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0ff25-292d-4433-a928-bf36b4ecab0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:29.198812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0ff25-292d-4433-a928-bf36b4ecab0f None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-79e0ff25-292d-4433-a928-bf36b4ecab0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:19:29.211898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:29.213088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:29.214203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:19:29.214203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:19:29.214203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:19:29.214203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:19:29.214203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:19:29.214203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:19:29.214203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:29.214203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:29.218020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 597/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:19:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:29.219403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e43010-c028-4e37-b81f-42a934db01b6 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:29.337347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e43010-c028-4e37-b81f-42a934db01b6 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:29.368399 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:29.384817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426f00e-93e9-4cde-b78a-72ef675b47ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:29.385748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426f00e-93e9-4cde-b78a-72ef675b47ed None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:29.386021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426f00e-93e9-4cde-b78a-72ef675b47ed None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:29.386317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426f00e-93e9-4cde-b78a-72ef675b47ed None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:29.388833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426f00e-93e9-4cde-b78a-72ef675b47ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:29.388833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426f00e-93e9-4cde-b78a-72ef675b47ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:29.388833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426f00e-93e9-4cde-b78a-72ef675b47ed None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:29.406544 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 621/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:19:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:29.412708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0ff25-292d-4433-a928-bf36b4ecab0f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:29.535965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0ff25-292d-4433-a928-bf36b4ecab0f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:29.565400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:29.572257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1e644-6b6e-4315-8889-6562649c0ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:29.572890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1e644-6b6e-4315-8889-6562649c0ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:29.573164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1e644-6b6e-4315-8889-6562649c0ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:29.573431 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1e644-6b6e-4315-8889-6562649c0ba2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:29.574153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1e644-6b6e-4315-8889-6562649c0ba2 None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=0a50fd417ae94859bb713b8b2066a9f4) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:29.574456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1e644-6b6e-4315-8889-6562649c0ba2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:29.575915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1e644-6b6e-4315-8889-6562649c0ba2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:29.582453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 598/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:19:29 2022] GET /identity/v3/role_assignments?user.id=0a50fd417ae94859bb713b8b2066a9f4&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:19:31.877583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f327198-a5b4-4ebd-b75b-8b8b12760030 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:31.878140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f327198-a5b4-4ebd-b75b-8b8b12760030 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:31.878140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f327198-a5b4-4ebd-b75b-8b8b12760030 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:31.879972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 622/1220] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:19:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:19:31.885997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832b2da-3fac-4896-96bd-2f11a9be016b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:31.886432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832b2da-3fac-4896-96bd-2f11a9be016b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:31.886537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832b2da-3fac-4896-96bd-2f11a9be016b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:31.909586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a832b2da-3fac-4896-96bd-2f11a9be016b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:19:31.911126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a832b2da-3fac-4896-96bd-2f11a9be016b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:19:31.981804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 599/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:19:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:19:32.310717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f327198-a5b4-4ebd-b75b-8b8b12760030 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:32.432825 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f327198-a5b4-4ebd-b75b-8b8b12760030 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:32.469236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81786fa8-e5c2-44c3-9e7d-0a22a4cb96ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:32.469862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81786fa8-e5c2-44c3-9e7d-0a22a4cb96ef None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:32.470175 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81786fa8-e5c2-44c3-9e7d-0a22a4cb96ef None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:32.470515 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81786fa8-e5c2-44c3-9e7d-0a22a4cb96ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:32.480712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81786fa8-e5c2-44c3-9e7d-0a22a4cb96ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-81786fa8-e5c2-44c3-9e7d-0a22a4cb96ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:19:32.485734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:32.486823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:32.487879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 623/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:19:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:32.493488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832b2da-3fac-4896-96bd-2f11a9be016b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:32.606846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832b2da-3fac-4896-96bd-2f11a9be016b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:32.636155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:32.648444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a7118-d3fd-4684-8c9c-8dede512b419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:32.649058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a7118-d3fd-4684-8c9c-8dede512b419 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:32.649336 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a7118-d3fd-4684-8c9c-8dede512b419 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:32.649604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a7118-d3fd-4684-8c9c-8dede512b419 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:32.650287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a7118-d3fd-4684-8c9c-8dede512b419 None admin] RBAC: Adding query filter params (name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:32.650587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a7118-d3fd-4684-8c9c-8dede512b419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:32.652008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a7118-d3fd-4684-8c9c-8dede512b419 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:32.661503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 600/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:19:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:19:32.667996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81786fa8-e5c2-44c3-9e7d-0a22a4cb96ef None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:32.717866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:32.780035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81786fa8-e5c2-44c3-9e7d-0a22a4cb96ef None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:32.815572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062a389-309f-4b5a-a748-4a672c75d971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:32.816193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a389-309f-4b5a-a748-4a672c75d971 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:32.817000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a389-309f-4b5a-a748-4a672c75d971 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:32.817267 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a389-309f-4b5a-a748-4a672c75d971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:32.826890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062a389-309f-4b5a-a748-4a672c75d971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:32.828334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062a389-309f-4b5a-a748-4a672c75d971 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-d062a389-309f-4b5a-a748-4a672c75d971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:32.843332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:32.844543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:32.850338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:32.850338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:32.850338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:32.850338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 624/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:32.851651 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a7118-d3fd-4684-8c9c-8dede512b419 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:32.962694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a7118-d3fd-4684-8c9c-8dede512b419 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:32.999235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a97f82-bf6b-4726-ab1b-3a0b3ee74af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:32.999856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a97f82-bf6b-4726-ab1b-3a0b3ee74af5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:33.000147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a97f82-bf6b-4726-ab1b-3a0b3ee74af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:33.000435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a97f82-bf6b-4726-ab1b-3a0b3ee74af5 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:33.001122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a97f82-bf6b-4726-ab1b-3a0b3ee74af5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:33.001454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a97f82-bf6b-4726-ab1b-3a0b3ee74af5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:33.002783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a97f82-bf6b-4726-ab1b-3a0b3ee74af5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:33.015135 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 601/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:33.022770 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062a389-309f-4b5a-a748-4a672c75d971 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:33.127556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:33.134958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062a389-309f-4b5a-a748-4a672c75d971 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:33.172431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01156ab0-f609-463c-9ca3-28f446913379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:33.173070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01156ab0-f609-463c-9ca3-28f446913379 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:33.173349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01156ab0-f609-463c-9ca3-28f446913379 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:33.173620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01156ab0-f609-463c-9ca3-28f446913379 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:33.181528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01156ab0-f609-463c-9ca3-28f446913379 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:33.182941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01156ab0-f609-463c-9ca3-28f446913379 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-01156ab0-f609-463c-9ca3-28f446913379 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:19:33.194186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:33.195216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:33.196479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:33.196479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:33.196479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:19:33.196479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:19:33.196479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:19:33.196479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:19:33.196479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:19:33.196479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:19:33.196479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:33.196479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:33.196479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 625/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:33.201391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a97f82-bf6b-4726-ab1b-3a0b3ee74af5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:33.251112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:33.313772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a97f82-bf6b-4726-ab1b-3a0b3ee74af5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:33.351673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9104a1e-3383-47cf-9c91-d83294ecf0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:33.352276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9104a1e-3383-47cf-9c91-d83294ecf0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:33.352587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9104a1e-3383-47cf-9c91-d83294ecf0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:33.352858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9104a1e-3383-47cf-9c91-d83294ecf0e7 None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:33.353582 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9104a1e-3383-47cf-9c91-d83294ecf0e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:33.353882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9104a1e-3383-47cf-9c91-d83294ecf0e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:33.355307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9104a1e-3383-47cf-9c91-d83294ecf0e7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:33.373924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 602/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 30 19:19:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:33.380186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01156ab0-f609-463c-9ca3-28f446913379 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:33.492831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01156ab0-f609-463c-9ca3-28f446913379 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:33.527519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba7d25-ac63-49ea-9283-9ca9deedabe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:33.528078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7d25-ac63-49ea-9283-9ca9deedabe4 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:33.528346 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7d25-ac63-49ea-9283-9ca9deedabe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:33.528644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7d25-ac63-49ea-9283-9ca9deedabe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:33.539350 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba7d25-ac63-49ea-9283-9ca9deedabe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:33.540833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba7d25-ac63-49ea-9283-9ca9deedabe4 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-a2ba7d25-ac63-49ea-9283-9ca9deedabe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:33.557070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:33.558454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:33.559814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:33.559814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:33.559814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:33.559814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 626/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:19:33.564108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9104a1e-3383-47cf-9c91-d83294ecf0e7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:33.676711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9104a1e-3383-47cf-9c91-d83294ecf0e7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:33.679042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:33.714820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0bc4c-13b8-40ab-a9f7-a4dc8d093a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:33.715421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0bc4c-13b8-40ab-a9f7-a4dc8d093a3b None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:33.715699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0bc4c-13b8-40ab-a9f7-a4dc8d093a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:33.715972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0bc4c-13b8-40ab-a9f7-a4dc8d093a3b None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:33.716666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0bc4c-13b8-40ab-a9f7-a4dc8d093a3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:33.716974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0bc4c-13b8-40ab-a9f7-a4dc8d093a3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:33.718337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0bc4c-13b8-40ab-a9f7-a4dc8d093a3b None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:33.732404 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 603/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:33.738544 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba7d25-ac63-49ea-9283-9ca9deedabe4 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:33.749669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:33.846836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba7d25-ac63-49ea-9283-9ca9deedabe4 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:33.880043 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f162a-2309-4d0d-ab70-ed63b2a03510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:33.880696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f162a-2309-4d0d-ab70-ed63b2a03510 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:33.880973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f162a-2309-4d0d-ab70-ed63b2a03510 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:33.881241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f162a-2309-4d0d-ab70-ed63b2a03510 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:33.889176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f162a-2309-4d0d-ab70-ed63b2a03510 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:33.890641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f162a-2309-4d0d-ab70-ed63b2a03510 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-857f162a-2309-4d0d-ab70-ed63b2a03510 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:19:33.902333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:33.903498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:33.904707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:19:33.904707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:19:33.904707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:19:33.904707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:19:33.904707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:19:33.904707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:19:33.904707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:33.904707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:33.904707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 627/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:19:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:33.911455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0bc4c-13b8-40ab-a9f7-a4dc8d093a3b None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:34.024570 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0bc4c-13b8-40ab-a9f7-a4dc8d093a3b None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:34.061328 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1b60b-1910-4eb3-a1a3-a966030d9412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:34.062038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b60b-1910-4eb3-a1a3-a966030d9412 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:34.062323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b60b-1910-4eb3-a1a3-a966030d9412 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:34.062627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b60b-1910-4eb3-a1a3-a966030d9412 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:34.063412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1b60b-1910-4eb3-a1a3-a966030d9412 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:34.063774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1b60b-1910-4eb3-a1a3-a966030d9412 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:34.065458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1b60b-1910-4eb3-a1a3-a966030d9412 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:34.079773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 604/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:19:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:19:34.086382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f162a-2309-4d0d-ab70-ed63b2a03510 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:34.158492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:34.197543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f162a-2309-4d0d-ab70-ed63b2a03510 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:34.238907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e8a95-6722-42ac-9f92-75f6430b8d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:34.239712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e8a95-6722-42ac-9f92-75f6430b8d8a None admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:34.240020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e8a95-6722-42ac-9f92-75f6430b8d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:34.240345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e8a95-6722-42ac-9f92-75f6430b8d8a None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/0a50fd417ae94859bb713b8b2066a9f4/roles/bd4e99b9d39b4ad2abf66671f7a35d61` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:34.256793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e8a95-6722-42ac-9f92-75f6430b8d8a None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=0a50fd417ae94859bb713b8b2066a9f4, role_id=bd4e99b9d39b4ad2abf66671f7a35d61)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:34.258366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e8a95-6722-42ac-9f92-75f6430b8d8a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:34.271486 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 628/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:19:34 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/0a50fd417ae94859bb713b8b2066a9f4/roles/bd4e99b9d39b4ad2abf66671f7a35d61 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:19:36.437732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6286fc-0617-4c27-b18a-359f0b426eb0 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:36.441969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6286fc-0617-4c27-b18a-359f0b426eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:36.442301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6286fc-0617-4c27-b18a-359f0b426eb0 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:36.443758 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 605/1233] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:19:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:19:36.450369 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e893aa8-d4fc-4c48-a0d8-12f16adf6bdd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:36.450862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e893aa8-d4fc-4c48-a0d8-12f16adf6bdd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:36.451059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e893aa8-d4fc-4c48-a0d8-12f16adf6bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:36.469854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6e893aa8-d4fc-4c48-a0d8-12f16adf6bdd None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:19:36.471053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6e893aa8-d4fc-4c48-a0d8-12f16adf6bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:19:36.543936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 629/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:19:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:19:36.854429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6286fc-0617-4c27-b18a-359f0b426eb0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:36.883266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:36.983918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6286fc-0617-4c27-b18a-359f0b426eb0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:37.023126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37498ec4-3468-4f65-8c4e-d7da8666395e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:37.023825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37498ec4-3468-4f65-8c4e-d7da8666395e None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:37.024158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37498ec4-3468-4f65-8c4e-d7da8666395e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:37.024468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37498ec4-3468-4f65-8c4e-d7da8666395e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:37.037000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37498ec4-3468-4f65-8c4e-d7da8666395e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-37498ec4-3468-4f65-8c4e-d7da8666395e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:19:37.042818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:37.048569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:37.049690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 606/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Sep 30 19:19:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:37.050816 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e893aa8-d4fc-4c48-a0d8-12f16adf6bdd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:37.103454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:37.162784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e893aa8-d4fc-4c48-a0d8-12f16adf6bdd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:37.197643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054dc222-3beb-4a5a-af0a-f8b63e50f7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:37.198249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054dc222-3beb-4a5a-af0a-f8b63e50f7ce None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:37.198525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054dc222-3beb-4a5a-af0a-f8b63e50f7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:37.198789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054dc222-3beb-4a5a-af0a-f8b63e50f7ce None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:37.199453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054dc222-3beb-4a5a-af0a-f8b63e50f7ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:37.199756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054dc222-3beb-4a5a-af0a-f8b63e50f7ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:37.201138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054dc222-3beb-4a5a-af0a-f8b63e50f7ce None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:37.209902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 630/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Sep 30 19:19:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:37.217025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37498ec4-3468-4f65-8c4e-d7da8666395e None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:37.311142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:37.329426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37498ec4-3468-4f65-8c4e-d7da8666395e None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:37.366477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6358096-ae5c-46b7-9f8c-3af8c802dee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:37.367207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6358096-ae5c-46b7-9f8c-3af8c802dee5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:37.367551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6358096-ae5c-46b7-9f8c-3af8c802dee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:37.367887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6358096-ae5c-46b7-9f8c-3af8c802dee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:37.379843 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6358096-ae5c-46b7-9f8c-3af8c802dee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:37.381535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6358096-ae5c-46b7-9f8c-3af8c802dee5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-d6358096-ae5c-46b7-9f8c-3af8c802dee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:37.398637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:37.399775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:37.405726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:37.405726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:37.405726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:37.405726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 607/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:37.406002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054dc222-3beb-4a5a-af0a-f8b63e50f7ce None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:37.510070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054dc222-3beb-4a5a-af0a-f8b63e50f7ce None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:37.519660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:37.549196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8bff8-6cce-43fd-aaa8-a5ba18b357d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:37.549820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8bff8-6cce-43fd-aaa8-a5ba18b357d5 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:37.550127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8bff8-6cce-43fd-aaa8-a5ba18b357d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:37.550958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8bff8-6cce-43fd-aaa8-a5ba18b357d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:37.551682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8bff8-6cce-43fd-aaa8-a5ba18b357d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:37.552075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8bff8-6cce-43fd-aaa8-a5ba18b357d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:37.553854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8bff8-6cce-43fd-aaa8-a5ba18b357d5 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:37.566054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 631/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:37.573263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6358096-ae5c-46b7-9f8c-3af8c802dee5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:37.683866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6358096-ae5c-46b7-9f8c-3af8c802dee5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:37.723151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028deee-0d93-45af-a7fb-2011fb92b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:37.724013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028deee-0d93-45af-a7fb-2011fb92b96c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:37.724394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028deee-0d93-45af-a7fb-2011fb92b96c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:37.724812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028deee-0d93-45af-a7fb-2011fb92b96c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:37.728389 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:37.735806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028deee-0d93-45af-a7fb-2011fb92b96c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:37.737671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028deee-0d93-45af-a7fb-2011fb92b96c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-4028deee-0d93-45af-a7fb-2011fb92b96c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:19:37.751400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:37.759281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:37.760700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:37.760700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:37.760700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:19:37.760700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:19:37.760700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:19:37.760700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:19:37.760700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:19:37.760700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:19:37.760700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:37.760700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:37.760700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 608/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:19:37.761546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8bff8-6cce-43fd-aaa8-a5ba18b357d5 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:37.869083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8bff8-6cce-43fd-aaa8-a5ba18b357d5 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:37.904864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db0736-0f91-4a2d-bbfc-8c347c7cb648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:37.905469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db0736-0f91-4a2d-bbfc-8c347c7cb648 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:37.905745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db0736-0f91-4a2d-bbfc-8c347c7cb648 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:37.906048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db0736-0f91-4a2d-bbfc-8c347c7cb648 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:37.906908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db0736-0f91-4a2d-bbfc-8c347c7cb648 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:37.907272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db0736-0f91-4a2d-bbfc-8c347c7cb648 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:37.908879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db0736-0f91-4a2d-bbfc-8c347c7cb648 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:37.930745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 632/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 30 19:19:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:37.936977 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028deee-0d93-45af-a7fb-2011fb92b96c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:38.044866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028deee-0d93-45af-a7fb-2011fb92b96c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:38.083406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fa2fc-6b36-42fd-8844-cc14d2d21bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:38.084152 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fa2fc-6b36-42fd-8844-cc14d2d21bfc None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:38.084786 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fa2fc-6b36-42fd-8844-cc14d2d21bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:38.085077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fa2fc-6b36-42fd-8844-cc14d2d21bfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:38.163302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:38.172015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fa2fc-6b36-42fd-8844-cc14d2d21bfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:38.173534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fa2fc-6b36-42fd-8844-cc14d2d21bfc None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:38.284176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 609/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:19:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:19:38.291206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db0736-0f91-4a2d-bbfc-8c347c7cb648 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:38.320743 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:38.399303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db0736-0f91-4a2d-bbfc-8c347c7cb648 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:38.435722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f753da-cb68-4614-a5b4-8998600f4489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:38.436508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f753da-cb68-4614-a5b4-8998600f4489 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:38.436789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f753da-cb68-4614-a5b4-8998600f4489 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:38.437054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f753da-cb68-4614-a5b4-8998600f4489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:38.448687 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f753da-cb68-4614-a5b4-8998600f4489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:38.450253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f753da-cb68-4614-a5b4-8998600f4489 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-c4f753da-cb68-4614-a5b4-8998600f4489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:38.467339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:38.468520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:38.469857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:38.469857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:38.469857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:38.469857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 633/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:19:38.475042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fa2fc-6b36-42fd-8844-cc14d2d21bfc None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:38.583888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:38.591365 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fa2fc-6b36-42fd-8844-cc14d2d21bfc None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:38.632922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991d042-7f7e-4ed2-bbe4-5e847b67dcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:38.633536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991d042-7f7e-4ed2-bbe4-5e847b67dcef None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:38.633813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991d042-7f7e-4ed2-bbe4-5e847b67dcef None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:38.634149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991d042-7f7e-4ed2-bbe4-5e847b67dcef None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:38.634795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991d042-7f7e-4ed2-bbe4-5e847b67dcef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:38.635123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991d042-7f7e-4ed2-bbe4-5e847b67dcef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:38.636765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991d042-7f7e-4ed2-bbe4-5e847b67dcef None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:38.648747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 610/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:38.654435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f753da-cb68-4614-a5b4-8998600f4489 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:38.741181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:38.763740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f753da-cb68-4614-a5b4-8998600f4489 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:38.798992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88516d6e-e5d8-49b9-9909-28f7dbdf8b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:38.799667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88516d6e-e5d8-49b9-9909-28f7dbdf8b7f None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:38.799944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88516d6e-e5d8-49b9-9909-28f7dbdf8b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:38.800272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88516d6e-e5d8-49b9-9909-28f7dbdf8b7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:38.808814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88516d6e-e5d8-49b9-9909-28f7dbdf8b7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:38.810404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88516d6e-e5d8-49b9-9909-28f7dbdf8b7f None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-88516d6e-e5d8-49b9-9909-28f7dbdf8b7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:19:38.822297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:38.823448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:38.824597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:19:38.824597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:19:38.824597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:19:38.824597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:19:38.824597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:19:38.824597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:19:38.824597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:38.824597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:38.824597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 634/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:19:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:38.829434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991d042-7f7e-4ed2-bbe4-5e847b67dcef None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:38.940485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991d042-7f7e-4ed2-bbe4-5e847b67dcef None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:38.980564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e1144-54c5-45e4-bcce-4c5f97236438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:38.981283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e1144-54c5-45e4-bcce-4c5f97236438 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:38.981584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e1144-54c5-45e4-bcce-4c5f97236438 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:38.981851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e1144-54c5-45e4-bcce-4c5f97236438 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:38.982544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e1144-54c5-45e4-bcce-4c5f97236438 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:38.982847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e1144-54c5-45e4-bcce-4c5f97236438 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:38.984469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e1144-54c5-45e4-bcce-4c5f97236438 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:38.998632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 611/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:19:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:39.005280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88516d6e-e5d8-49b9-9909-28f7dbdf8b7f None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:39.131065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88516d6e-e5d8-49b9-9909-28f7dbdf8b7f None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:39.173097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c4e55-5522-4bee-af72-b182a993d788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:39.173829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c4e55-5522-4bee-af72-b182a993d788 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:39.174100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c4e55-5522-4bee-af72-b182a993d788 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:39.174364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c4e55-5522-4bee-af72-b182a993d788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:39.175074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c4e55-5522-4bee-af72-b182a993d788 None admin] RBAC: Adding query filter params (role.id=bd4e99b9d39b4ad2abf66671f7a35d61, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=0a50fd417ae94859bb713b8b2066a9f4) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:39.175371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c4e55-5522-4bee-af72-b182a993d788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:39.177173 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c4e55-5522-4bee-af72-b182a993d788 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:39.184287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 635/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:19:39 2022] GET /identity/v3/role_assignments?user.id=0a50fd417ae94859bb713b8b2066a9f4&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=bd4e99b9d39b4ad2abf66671f7a35d61 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:41.415726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b86df1-5b2a-476b-a82d-d5ff3c85ffb3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:41.416249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b86df1-5b2a-476b-a82d-d5ff3c85ffb3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:41.416249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b86df1-5b2a-476b-a82d-d5ff3c85ffb3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:41.417654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 612/1247] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:19:41.423398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d245f-ea20-462a-82fc-5f501460734c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:41.423936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d245f-ea20-462a-82fc-5f501460734c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:41.423936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d245f-ea20-462a-82fc-5f501460734c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:41.430208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:41.446123 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-813d245f-ea20-462a-82fc-5f501460734c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:19:41.447332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-813d245f-ea20-462a-82fc-5f501460734c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:19:41.516431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 636/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:19:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 19:19:41.802996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b86df1-5b2a-476b-a82d-d5ff3c85ffb3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:41.823245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:41.910694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b86df1-5b2a-476b-a82d-d5ff3c85ffb3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:41.959969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e35c3d-b766-45ab-8343-4d70f5080a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:41.960631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e35c3d-b766-45ab-8343-4d70f5080a5f None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:41.960914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e35c3d-b766-45ab-8343-4d70f5080a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:41.961181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e35c3d-b766-45ab-8343-4d70f5080a5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:41.971551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e35c3d-b766-45ab-8343-4d70f5080a5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-67e35c3d-b766-45ab-8343-4d70f5080a5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:19:41.976798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:41.977987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:41.979042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 613/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:19:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:41.983952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d245f-ea20-462a-82fc-5f501460734c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:42.086271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d245f-ea20-462a-82fc-5f501460734c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:42.118582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:42.123909 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e435bd-29ca-448f-a6a3-64dd8e555133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:42.124546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e435bd-29ca-448f-a6a3-64dd8e555133 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:42.124822 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e435bd-29ca-448f-a6a3-64dd8e555133 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:42.125089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e435bd-29ca-448f-a6a3-64dd8e555133 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:42.125745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e435bd-29ca-448f-a6a3-64dd8e555133 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:42.126040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e435bd-29ca-448f-a6a3-64dd8e555133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:42.127410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e435bd-29ca-448f-a6a3-64dd8e555133 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:42.137064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 637/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:19:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:42.143379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e35c3d-b766-45ab-8343-4d70f5080a5f None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:42.223580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:42.247177 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e35c3d-b766-45ab-8343-4d70f5080a5f None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:42.283748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346e8d0-5d89-4d02-a886-3f3a37957fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:42.284391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e8d0-5d89-4d02-a886-3f3a37957fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:42.284687 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e8d0-5d89-4d02-a886-3f3a37957fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:42.285567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346e8d0-5d89-4d02-a886-3f3a37957fa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:42.296339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346e8d0-5d89-4d02-a886-3f3a37957fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:42.297863 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346e8d0-5d89-4d02-a886-3f3a37957fa1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-f346e8d0-5d89-4d02-a886-3f3a37957fa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:42.312957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:42.314075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:42.315012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:42.315012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:42.315012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:42.315012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 614/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:42.320021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e435bd-29ca-448f-a6a3-64dd8e555133 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:42.429604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e435bd-29ca-448f-a6a3-64dd8e555133 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:42.468173 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e5fe6-9244-4765-bfe0-96f58da3dd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:42.468952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e5fe6-9244-4765-bfe0-96f58da3dd47 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:42.469230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e5fe6-9244-4765-bfe0-96f58da3dd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:42.469501 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e5fe6-9244-4765-bfe0-96f58da3dd47 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:42.470180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e5fe6-9244-4765-bfe0-96f58da3dd47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:42.470598 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e5fe6-9244-4765-bfe0-96f58da3dd47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:42.472826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e5fe6-9244-4765-bfe0-96f58da3dd47 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:42.492412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 638/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:42.494777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346e8d0-5d89-4d02-a886-3f3a37957fa1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:42.607966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346e8d0-5d89-4d02-a886-3f3a37957fa1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:42.636172 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:42.645190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f129429-31a3-410f-a5fa-e997555a340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:42.645819 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f129429-31a3-410f-a5fa-e997555a340a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:42.646115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f129429-31a3-410f-a5fa-e997555a340a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:42.646383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f129429-31a3-410f-a5fa-e997555a340a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:42.654282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f129429-31a3-410f-a5fa-e997555a340a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:42.655651 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f129429-31a3-410f-a5fa-e997555a340a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-0f129429-31a3-410f-a5fa-e997555a340a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:19:42.668831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:42.670083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:42.671258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:42.671258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:42.671258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:19:42.671258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:19:42.671258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:19:42.671258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:19:42.671258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:19:42.671258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:19:42.671258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:42.671258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:42.671258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 615/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:19:42.676312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e5fe6-9244-4765-bfe0-96f58da3dd47 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:42.733274 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:42.779309 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e5fe6-9244-4765-bfe0-96f58da3dd47 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:42.814334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd080f4-4eac-4ce6-9eb3-1451b6f1301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:42.814950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd080f4-4eac-4ce6-9eb3-1451b6f1301a None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:42.815224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd080f4-4eac-4ce6-9eb3-1451b6f1301a None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:42.815491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd080f4-4eac-4ce6-9eb3-1451b6f1301a None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:42.816844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd080f4-4eac-4ce6-9eb3-1451b6f1301a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:42.817145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd080f4-4eac-4ce6-9eb3-1451b6f1301a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:42.818607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd080f4-4eac-4ce6-9eb3-1451b6f1301a None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:42.836864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 639/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 30 19:19:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:42.842655 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f129429-31a3-410f-a5fa-e997555a340a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:42.945418 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f129429-31a3-410f-a5fa-e997555a340a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:42.980241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71df9e4-7f3b-4560-ad5b-3bcdb074cbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:42.980876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71df9e4-7f3b-4560-ad5b-3bcdb074cbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:42.981162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71df9e4-7f3b-4560-ad5b-3bcdb074cbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:42.981432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71df9e4-7f3b-4560-ad5b-3bcdb074cbe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:43.038506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:43.051682 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71df9e4-7f3b-4560-ad5b-3bcdb074cbe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:43.053290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71df9e4-7f3b-4560-ad5b-3bcdb074cbe5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:43.168760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 616/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:19:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:19:43.174788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd080f4-4eac-4ce6-9eb3-1451b6f1301a None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:43.273601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd080f4-4eac-4ce6-9eb3-1451b6f1301a None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:43.285995 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:43.306814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3479ac-aa21-4d57-99ab-de71bda354c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:43.307431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3479ac-aa21-4d57-99ab-de71bda354c7 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:43.307708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3479ac-aa21-4d57-99ab-de71bda354c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:43.308001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3479ac-aa21-4d57-99ab-de71bda354c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:43.318105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3479ac-aa21-4d57-99ab-de71bda354c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:43.319532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3479ac-aa21-4d57-99ab-de71bda354c7 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-5b3479ac-aa21-4d57-99ab-de71bda354c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:43.334113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:43.335238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:43.336336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:43.336336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:43.336336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:43.336336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 640/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:43.340967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71df9e4-7f3b-4560-ad5b-3bcdb074cbe5 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:43.424546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:43.436295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71df9e4-7f3b-4560-ad5b-3bcdb074cbe5 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:43.469444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5e1e3-f39e-49eb-ab56-a38ef344abf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:43.470034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5e1e3-f39e-49eb-ab56-a38ef344abf1 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:43.470311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5e1e3-f39e-49eb-ab56-a38ef344abf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:43.470582 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5e1e3-f39e-49eb-ab56-a38ef344abf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:43.471216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5e1e3-f39e-49eb-ab56-a38ef344abf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:43.471512 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5e1e3-f39e-49eb-ab56-a38ef344abf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:43.472940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5e1e3-f39e-49eb-ab56-a38ef344abf1 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:43.483693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 617/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:43.489986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3479ac-aa21-4d57-99ab-de71bda354c7 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:43.584806 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3479ac-aa21-4d57-99ab-de71bda354c7 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:43.616216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6e5ae-2854-4064-af4d-ebd52e4a53fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:43.616874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6e5ae-2854-4064-af4d-ebd52e4a53fc None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:43.617149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6e5ae-2854-4064-af4d-ebd52e4a53fc None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:43.617415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6e5ae-2854-4064-af4d-ebd52e4a53fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:43.623674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6e5ae-2854-4064-af4d-ebd52e4a53fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:43.625136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd6e5ae-2854-4064-af4d-ebd52e4a53fc None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-bbd6e5ae-2854-4064-af4d-ebd52e4a53fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:19:43.636750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:43.637803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:43.639000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:19:43.639000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:19:43.639000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:19:43.639000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:19:43.639000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:19:43.639000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:19:43.639000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:43.639000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:43.640521 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 641/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:19:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:43.643577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5e1e3-f39e-49eb-ab56-a38ef344abf1 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:43.740393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5e1e3-f39e-49eb-ab56-a38ef344abf1 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:43.772265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02aaf6-10b5-4f81-8e1f-b39a1b21ba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:43.772872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02aaf6-10b5-4f81-8e1f-b39a1b21ba92 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:43.773149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02aaf6-10b5-4f81-8e1f-b39a1b21ba92 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:43.773417 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02aaf6-10b5-4f81-8e1f-b39a1b21ba92 None admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:43.774097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02aaf6-10b5-4f81-8e1f-b39a1b21ba92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:43.774397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02aaf6-10b5-4f81-8e1f-b39a1b21ba92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:43.775809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02aaf6-10b5-4f81-8e1f-b39a1b21ba92 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:43.787295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 618/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:19:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:19:43.793407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6e5ae-2854-4064-af4d-ebd52e4a53fc None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:43.818887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:43.889048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6e5ae-2854-4064-af4d-ebd52e4a53fc None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:43.921081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d9fb4-9a0b-46e5-8b5f-55ceddff613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:43.921663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d9fb4-9a0b-46e5-8b5f-55ceddff613b None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:43.921938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d9fb4-9a0b-46e5-8b5f-55ceddff613b None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:43.922203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d9fb4-9a0b-46e5-8b5f-55ceddff613b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:43.922908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d9fb4-9a0b-46e5-8b5f-55ceddff613b None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=0a50fd417ae94859bb713b8b2066a9f4) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:43.923207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d9fb4-9a0b-46e5-8b5f-55ceddff613b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:43.924640 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d9fb4-9a0b-46e5-8b5f-55ceddff613b None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:43.929735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 642/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:19:43 2022] GET /identity/v3/role_assignments?user.id=0a50fd417ae94859bb713b8b2066a9f4&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:46.016031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f02ccc-e48d-49e8-9748-58f59ea0a899 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:46.016580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f02ccc-e48d-49e8-9748-58f59ea0a899 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:46.016580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f02ccc-e48d-49e8-9748-58f59ea0a899 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:46.017933 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 619/1261] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:19:46.023010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57aca9-3d09-43c6-9042-ec1e6dd1ca5b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:46.023421 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57aca9-3d09-43c6-9042-ec1e6dd1ca5b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:46.023562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57aca9-3d09-43c6-9042-ec1e6dd1ca5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:46.041342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1c57aca9-3d09-43c6-9042-ec1e6dd1ca5b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:19:46.042501 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1c57aca9-3d09-43c6-9042-ec1e6dd1ca5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:19:46.113724 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 643/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:19:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:19:46.378341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f02ccc-e48d-49e8-9748-58f59ea0a899 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:46.389133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:46.477451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f02ccc-e48d-49e8-9748-58f59ea0a899 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:46.509878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f2b23-6a16-4559-b444-e41b9b54c4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:46.510486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f2b23-6a16-4559-b444-e41b9b54c4fb None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:46.510762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f2b23-6a16-4559-b444-e41b9b54c4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:46.511029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f2b23-6a16-4559-b444-e41b9b54c4fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:46.521255 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f2b23-6a16-4559-b444-e41b9b54c4fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-648f2b23-6a16-4559-b444-e41b9b54c4fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:19:46.525922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:46.527025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:46.527934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 620/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:19:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:46.533111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57aca9-3d09-43c6-9042-ec1e6dd1ca5b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:46.551342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:46.629067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57aca9-3d09-43c6-9042-ec1e6dd1ca5b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:46.661221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced7fc7-b36f-4f9e-9695-3b1d12ae5d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:46.661807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced7fc7-b36f-4f9e-9695-3b1d12ae5d29 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:46.662079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced7fc7-b36f-4f9e-9695-3b1d12ae5d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:46.662343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced7fc7-b36f-4f9e-9695-3b1d12ae5d29 None admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:46.662991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced7fc7-b36f-4f9e-9695-3b1d12ae5d29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:46.663284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced7fc7-b36f-4f9e-9695-3b1d12ae5d29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:46.664908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced7fc7-b36f-4f9e-9695-3b1d12ae5d29 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:46.672456 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 644/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:19:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:46.678704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f2b23-6a16-4559-b444-e41b9b54c4fb None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:46.742433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:46.774288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f2b23-6a16-4559-b444-e41b9b54c4fb None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:46.807407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e171e-8dff-4669-8ea4-a6d71c2bb669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:46.808013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e171e-8dff-4669-8ea4-a6d71c2bb669 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:46.808310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e171e-8dff-4669-8ea4-a6d71c2bb669 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:46.808580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e171e-8dff-4669-8ea4-a6d71c2bb669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:46.818720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e171e-8dff-4669-8ea4-a6d71c2bb669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:46.820156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e171e-8dff-4669-8ea4-a6d71c2bb669 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-7d5e171e-8dff-4669-8ea4-a6d71c2bb669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:46.834163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:46.835356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:46.836533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:46.836533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:46.836533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:46.836533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 621/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:46.840956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced7fc7-b36f-4f9e-9695-3b1d12ae5d29 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:46.918523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:46.936773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced7fc7-b36f-4f9e-9695-3b1d12ae5d29 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:46.967912 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1ad58-15e5-46d3-9d8e-303e39f5bbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:46.968523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1ad58-15e5-46d3-9d8e-303e39f5bbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:46.968797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1ad58-15e5-46d3-9d8e-303e39f5bbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:46.969064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1ad58-15e5-46d3-9d8e-303e39f5bbc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:46.969684 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1ad58-15e5-46d3-9d8e-303e39f5bbc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:46.969976 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1ad58-15e5-46d3-9d8e-303e39f5bbc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:46.971307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1ad58-15e5-46d3-9d8e-303e39f5bbc3 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:46.982753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 645/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:46.988856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e171e-8dff-4669-8ea4-a6d71c2bb669 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:47.084801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e171e-8dff-4669-8ea4-a6d71c2bb669 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:47.104858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:47.119417 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54618d8-5ce8-496d-80ae-ce0e325f2967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:47.120035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54618d8-5ce8-496d-80ae-ce0e325f2967 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:47.120310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54618d8-5ce8-496d-80ae-ce0e325f2967 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:47.120608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54618d8-5ce8-496d-80ae-ce0e325f2967 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:47.127736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54618d8-5ce8-496d-80ae-ce0e325f2967 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:47.129189 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54618d8-5ce8-496d-80ae-ce0e325f2967 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-e54618d8-5ce8-496d-80ae-ce0e325f2967 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:19:47.139925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:47.141069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:47.142091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:47.142091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:47.142091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:19:47.142091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:19:47.142091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:19:47.142091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:19:47.142091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:19:47.142091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:19:47.142091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:47.142091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:47.142091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 622/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:47.146922 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1ad58-15e5-46d3-9d8e-303e39f5bbc3 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:47.241986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1ad58-15e5-46d3-9d8e-303e39f5bbc3 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:47.279921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af7098-f6e3-43fd-8d7f-638d29e21ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:47.280541 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af7098-f6e3-43fd-8d7f-638d29e21ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:47.280814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af7098-f6e3-43fd-8d7f-638d29e21ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:47.281080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af7098-f6e3-43fd-8d7f-638d29e21ef8 None admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:47.281789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af7098-f6e3-43fd-8d7f-638d29e21ef8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:47.282091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af7098-f6e3-43fd-8d7f-638d29e21ef8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:47.283487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af7098-f6e3-43fd-8d7f-638d29e21ef8 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:47.292583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:47.301719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 646/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 30 19:19:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:47.307575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54618d8-5ce8-496d-80ae-ce0e325f2967 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:47.403231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54618d8-5ce8-496d-80ae-ce0e325f2967 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:47.435448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d8675-38c2-41c0-ad0d-96695941e009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:47.436048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d8675-38c2-41c0-ad0d-96695941e009 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:47.436321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d8675-38c2-41c0-ad0d-96695941e009 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:47.436612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d8675-38c2-41c0-ad0d-96695941e009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:47.447242 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d8675-38c2-41c0-ad0d-96695941e009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:47.448680 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d8675-38c2-41c0-ad0d-96695941e009 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bb4d8675-38c2-41c0-ad0d-96695941e009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:47.462182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:47.463295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:47.464600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:47.464600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:47.464600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:47.464600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 623/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:47.469011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af7098-f6e3-43fd-8d7f-638d29e21ef8 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:47.563581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af7098-f6e3-43fd-8d7f-638d29e21ef8 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:47.596196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eaa3b-c0de-46ba-bf67-39fc93fc7cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:47.596812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eaa3b-c0de-46ba-bf67-39fc93fc7cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:47.597090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eaa3b-c0de-46ba-bf67-39fc93fc7cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:47.597910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eaa3b-c0de-46ba-bf67-39fc93fc7cc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:47.598541 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eaa3b-c0de-46ba-bf67-39fc93fc7cc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:47.598833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eaa3b-c0de-46ba-bf67-39fc93fc7cc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:47.600169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eaa3b-c0de-46ba-bf67-39fc93fc7cc2 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:47.610776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 647/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:47.616500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d8675-38c2-41c0-ad0d-96695941e009 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:47.647480 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:47.712891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d8675-38c2-41c0-ad0d-96695941e009 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:47.745022 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e1a3c-5c73-4382-9101-b2c67fd7c9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:47.745620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e1a3c-5c73-4382-9101-b2c67fd7c9df None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:47.745893 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e1a3c-5c73-4382-9101-b2c67fd7c9df None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:47.746162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e1a3c-5c73-4382-9101-b2c67fd7c9df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:47.752434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e1a3c-5c73-4382-9101-b2c67fd7c9df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:47.753847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e1a3c-5c73-4382-9101-b2c67fd7c9df None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-ba8e1a3c-5c73-4382-9101-b2c67fd7c9df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:19:47.764537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:47.765593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:47.766746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:19:47.766746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:19:47.766746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:19:47.766746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:19:47.766746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:19:47.766746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:19:47.766746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:47.766746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:47.766746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 624/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:19:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:47.771308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eaa3b-c0de-46ba-bf67-39fc93fc7cc2 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:47.827165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:47.867642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eaa3b-c0de-46ba-bf67-39fc93fc7cc2 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:47.899453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6a342-fdce-49a7-acd5-56d7dc9aa2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:47.900044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6a342-fdce-49a7-acd5-56d7dc9aa2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:47.900319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6a342-fdce-49a7-acd5-56d7dc9aa2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:47.900614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6a342-fdce-49a7-acd5-56d7dc9aa2b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:47.901293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6a342-fdce-49a7-acd5-56d7dc9aa2b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:47.901590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6a342-fdce-49a7-acd5-56d7dc9aa2b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:47.902990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6a342-fdce-49a7-acd5-56d7dc9aa2b1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:47.915139 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 648/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:19:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:47.921143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e1a3c-5c73-4382-9101-b2c67fd7c9df None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:48.009468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:48.018020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e1a3c-5c73-4382-9101-b2c67fd7c9df None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:48.050967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221838f9-8d2a-4293-9a09-966974a0a140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:48.051603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221838f9-8d2a-4293-9a09-966974a0a140 None admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:48.051880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221838f9-8d2a-4293-9a09-966974a0a140 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:48.052149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221838f9-8d2a-4293-9a09-966974a0a140 None admin] PATH_INFO: `/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/0a50fd417ae94859bb713b8b2066a9f4/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:48.054021 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221838f9-8d2a-4293-9a09-966974a0a140 None admin] RBAC: Authorizing `identity:create_grant(project_id=e761ae9e97264607ad7eab7dc2e64b00, user_id=0a50fd417ae94859bb713b8b2066a9f4, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:48.055452 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221838f9-8d2a-4293-9a09-966974a0a140 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:48.067305 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 625/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Sep 30 19:19:47 2022] PUT /identity/v3/projects/e761ae9e97264607ad7eab7dc2e64b00/users/0a50fd417ae94859bb713b8b2066a9f4/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:19:50.197479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3c6a5-7760-49ab-b42b-e34cba75a364 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:50.198064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3c6a5-7760-49ab-b42b-e34cba75a364 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:50.198064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3c6a5-7760-49ab-b42b-e34cba75a364 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:50.199366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 649/1274] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:19:50.205627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44394a-2863-4b49-97aa-6069c5723cfc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:50.206045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44394a-2863-4b49-97aa-6069c5723cfc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:50.206286 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44394a-2863-4b49-97aa-6069c5723cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:50.225310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7a44394a-2863-4b49-97aa-6069c5723cfc None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:19:50.226475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7a44394a-2863-4b49-97aa-6069c5723cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:19:50.289467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 626/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:19:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:19:50.551880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3c6a5-7760-49ab-b42b-e34cba75a364 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:50.648349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3c6a5-7760-49ab-b42b-e34cba75a364 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:50.665233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:50.681757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e72ec1-8960-4e11-8a79-e428b6a5c60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:50.682368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72ec1-8960-4e11-8a79-e428b6a5c60c None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:50.682644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72ec1-8960-4e11-8a79-e428b6a5c60c None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:50.682912 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72ec1-8960-4e11-8a79-e428b6a5c60c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:50.692490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e72ec1-8960-4e11-8a79-e428b6a5c60c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-76e72ec1-8960-4e11-8a79-e428b6a5c60c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 19:19:50.697192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:50.698126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:50.699052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 650/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Sep 30 19:19:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:50.704008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44394a-2863-4b49-97aa-6069c5723cfc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:50.781032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:50.799506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44394a-2863-4b49-97aa-6069c5723cfc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:50.832979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba73a9-141a-4625-a37e-17c7bfb21263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:50.833509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba73a9-141a-4625-a37e-17c7bfb21263 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:50.833783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba73a9-141a-4625-a37e-17c7bfb21263 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:50.834049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba73a9-141a-4625-a37e-17c7bfb21263 None admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:50.834773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba73a9-141a-4625-a37e-17c7bfb21263 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:50.835072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba73a9-141a-4625-a37e-17c7bfb21263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:50.836504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba73a9-141a-4625-a37e-17c7bfb21263 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:50.844146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 627/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Sep 30 19:19:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:50.850513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e72ec1-8960-4e11-8a79-e428b6a5c60c None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:50.945568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e72ec1-8960-4e11-8a79-e428b6a5c60c None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:50.977690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0005ae9-0864-4906-9d84-3c3b2fc52e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:50.978293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0005ae9-0864-4906-9d84-3c3b2fc52e01 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:50.978568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0005ae9-0864-4906-9d84-3c3b2fc52e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:50.978835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0005ae9-0864-4906-9d84-3c3b2fc52e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:50.988870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0005ae9-0864-4906-9d84-3c3b2fc52e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:50.990278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0005ae9-0864-4906-9d84-3c3b2fc52e01 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-f0005ae9-0864-4906-9d84-3c3b2fc52e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:51.004101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:51.005254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:51.006285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:51.006285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:51.006285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:51.006285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 651/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 19:19:51.010652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba73a9-141a-4625-a37e-17c7bfb21263 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:51.113139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba73a9-141a-4625-a37e-17c7bfb21263 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:51.149979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d7404-1ed0-4c05-9611-e1e86288b20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:51.150779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d7404-1ed0-4c05-9611-e1e86288b20a None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:51.151122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d7404-1ed0-4c05-9611-e1e86288b20a None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:51.151483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d7404-1ed0-4c05-9611-e1e86288b20a None admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:51.152342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d7404-1ed0-4c05-9611-e1e86288b20a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:51.152795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d7404-1ed0-4c05-9611-e1e86288b20a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:51.154564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d7404-1ed0-4c05-9611-e1e86288b20a None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:51.167362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 628/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:51.173080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0005ae9-0864-4906-9d84-3c3b2fc52e01 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:51.203341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:51.275994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0005ae9-0864-4906-9d84-3c3b2fc52e01 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:51.307842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45234b5-58bf-4b0a-840d-f5e836f775de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:51.308479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45234b5-58bf-4b0a-840d-f5e836f775de None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:51.308753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45234b5-58bf-4b0a-840d-f5e836f775de None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:51.309019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45234b5-58bf-4b0a-840d-f5e836f775de None admin] PATH_INFO: `/v3/users/placement` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:51.315949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45234b5-58bf-4b0a-840d-f5e836f775de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:51.317376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45234b5-58bf-4b0a-840d-f5e836f775de None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-e45234b5-58bf-4b0a-840d-f5e836f775de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 19:19:51.327876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:51.329012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:51.330209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:51.330209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:51.330209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 19:19:51.330209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 19:19:51.330209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 19:19:51.330209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 19:19:51.330209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 19:19:51.330209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 19:19:51.330209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 19:19:51.330209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:51.330209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 652/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:51.335422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d7404-1ed0-4c05-9611-e1e86288b20a None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:51.339109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:51.431505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d7404-1ed0-4c05-9611-e1e86288b20a None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:51.464256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bce8e-3ac2-4cd9-b717-6733e24caaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:51.464878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bce8e-3ac2-4cd9-b717-6733e24caaea None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:51.465157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bce8e-3ac2-4cd9-b717-6733e24caaea None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:51.465426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bce8e-3ac2-4cd9-b717-6733e24caaea None admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:51.466140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bce8e-3ac2-4cd9-b717-6733e24caaea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:51.466439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bce8e-3ac2-4cd9-b717-6733e24caaea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:51.467855 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bce8e-3ac2-4cd9-b717-6733e24caaea None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:51.484105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 629/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Sep 30 19:19:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:51.490001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45234b5-58bf-4b0a-840d-f5e836f775de None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:51.577134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:51.587294 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45234b5-58bf-4b0a-840d-f5e836f775de None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:51.618550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c10ae-6ffd-428a-9c48-766f0822c581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:51.619144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c10ae-6ffd-428a-9c48-766f0822c581 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:51.619417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c10ae-6ffd-428a-9c48-766f0822c581 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:51.619687 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c10ae-6ffd-428a-9c48-766f0822c581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:51.682869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c10ae-6ffd-428a-9c48-766f0822c581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:51.684233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c10ae-6ffd-428a-9c48-766f0822c581 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:51.766420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:51.776675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 653/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Sep 30 19:19:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 19:19:51.782418 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bce8e-3ac2-4cd9-b717-6733e24caaea None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:51.841247 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:51.878139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bce8e-3ac2-4cd9-b717-6733e24caaea None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:51.910997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48e76e-223a-4e8f-a7c3-9a3a3568a3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:51.911609 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48e76e-223a-4e8f-a7c3-9a3a3568a3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:51.911884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48e76e-223a-4e8f-a7c3-9a3a3568a3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:51.912153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48e76e-223a-4e8f-a7c3-9a3a3568a3f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:51.921942 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48e76e-223a-4e8f-a7c3-9a3a3568a3f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:51.923375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48e76e-223a-4e8f-a7c3-9a3a3568a3f7 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-bf48e76e-223a-4e8f-a7c3-9a3a3568a3f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 19:19:51.937838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:51.938965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 19:19:51.940096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 19:19:51.940096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 19:19:51.940096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:51.940096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 630/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:19:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:51.944609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c10ae-6ffd-428a-9c48-766f0822c581 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:52.038111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c10ae-6ffd-428a-9c48-766f0822c581 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:52.070506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8743e-5497-468d-87e5-9ef6ce8e3e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:52.071089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8743e-5497-468d-87e5-9ef6ce8e3e3d None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:52.071362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8743e-5497-468d-87e5-9ef6ce8e3e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:52.071629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8743e-5497-468d-87e5-9ef6ce8e3e3d None admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:52.072256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8743e-5497-468d-87e5-9ef6ce8e3e3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:52.072581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8743e-5497-468d-87e5-9ef6ce8e3e3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:52.074031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8743e-5497-468d-87e5-9ef6ce8e3e3d None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:52.086594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 654/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Sep 30 19:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:52.092048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48e76e-223a-4e8f-a7c3-9a3a3568a3f7 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:52.187425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48e76e-223a-4e8f-a7c3-9a3a3568a3f7 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:52.200768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:52.220726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cdcd6-6658-48f7-b0ba-a5e8b35a29f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:52.221331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cdcd6-6658-48f7-b0ba-a5e8b35a29f2 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:52.221603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cdcd6-6658-48f7-b0ba-a5e8b35a29f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:52.221865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cdcd6-6658-48f7-b0ba-a5e8b35a29f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:52.228224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377cdcd6-6658-48f7-b0ba-a5e8b35a29f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:52.229669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377cdcd6-6658-48f7-b0ba-a5e8b35a29f2 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application [None req-377cdcd6-6658-48f7-b0ba-a5e8b35a29f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 19:19:52.240499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:52.241541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:52.242610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 19:19:52.242610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 19:19:52.242610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 19:19:52.242610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 19:19:52.242610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 19:19:52.242610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 19:19:52.242610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 19:19:52.242610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: ERROR keystone.server.flask.application Sep 30 19:19:52.242610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 631/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Sep 30 19:19:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:52.247175 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8743e-5497-468d-87e5-9ef6ce8e3e3d None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:52.295735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:52.343818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8743e-5497-468d-87e5-9ef6ce8e3e3d None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:52.375312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c099d-a240-4ee0-b7c3-c3c9b7cdf6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:52.375898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c099d-a240-4ee0-b7c3-c3c9b7cdf6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:52.376170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c099d-a240-4ee0-b7c3-c3c9b7cdf6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:52.376464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c099d-a240-4ee0-b7c3-c3c9b7cdf6d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:52.377136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c099d-a240-4ee0-b7c3-c3c9b7cdf6d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:52.377434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c099d-a240-4ee0-b7c3-c3c9b7cdf6d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:52.379376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c099d-a240-4ee0-b7c3-c3c9b7cdf6d6 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:52.391519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 655/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Sep 30 19:19:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:52.398103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cdcd6-6658-48f7-b0ba-a5e8b35a29f2 None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:52.494981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cdcd6-6658-48f7-b0ba-a5e8b35a29f2 None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:52.526550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141ddfd-c2c5-4fe3-8d9f-b670c23af1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:52.527129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141ddfd-c2c5-4fe3-8d9f-b670c23af1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:52.527401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141ddfd-c2c5-4fe3-8d9f-b670c23af1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:52.527661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141ddfd-c2c5-4fe3-8d9f-b670c23af1d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:52.528386 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141ddfd-c2c5-4fe3-8d9f-b670c23af1d5 None admin] RBAC: Adding query filter params (role.id=6c2982cfde074f068473596a4f4e61c2, scope.project.id=e761ae9e97264607ad7eab7dc2e64b00, user.id=0a50fd417ae94859bb713b8b2066a9f4) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:52.528690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141ddfd-c2c5-4fe3-8d9f-b670c23af1d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:52.530088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141ddfd-c2c5-4fe3-8d9f-b670c23af1d5 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:52.535545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 632/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Sep 30 19:19:52 2022] GET /identity/v3/role_assignments?user.id=0a50fd417ae94859bb713b8b2066a9f4&scope.project.id=e761ae9e97264607ad7eab7dc2e64b00&role.id=6c2982cfde074f068473596a4f4e61c2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:54.665417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a93f2a-5263-4aaf-af1f-b8ce2b2de5c8 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:54.665922 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a93f2a-5263-4aaf-af1f-b8ce2b2de5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:54.665922 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a93f2a-5263-4aaf-af1f-b8ce2b2de5c8 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:54.667204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 656/1288] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:19:54.672357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8e2eb-3a60-4fe1-b9f6-2738a130d055 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:54.672819 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8e2eb-3a60-4fe1-b9f6-2738a130d055 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:54.672916 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8e2eb-3a60-4fe1-b9f6-2738a130d055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:54.691185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-12e8e2eb-3a60-4fe1-b9f6-2738a130d055 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:19:54.692383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-12e8e2eb-3a60-4fe1-b9f6-2738a130d055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:19:54.720190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:54.756016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 633/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:19:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:19:55.019277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a93f2a-5263-4aaf-af1f-b8ce2b2de5c8 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:55.119993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a93f2a-5263-4aaf-af1f-b8ce2b2de5c8 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:55.132218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:55.158080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834f5a9-f731-441c-b7ff-226daa63ea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:55.158700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834f5a9-f731-441c-b7ff-226daa63ea13 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:55.158976 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834f5a9-f731-441c-b7ff-226daa63ea13 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:55.159244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834f5a9-f731-441c-b7ff-226daa63ea13 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:55.159811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834f5a9-f731-441c-b7ff-226daa63ea13 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:55.161193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834f5a9-f731-441c-b7ff-226daa63ea13 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-6834f5a9-f731-441c-b7ff-226daa63ea13 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:19:55.169846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:19:55.171114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:19:55.172284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 657/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 30 19:19:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:19:55.177041 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8e2eb-3a60-4fe1-b9f6-2738a130d055 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:55.272087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8e2eb-3a60-4fe1-b9f6-2738a130d055 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:55.304553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612895c8-9eb6-4651-b501-00bf3f2f39e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:55.305147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612895c8-9eb6-4651-b501-00bf3f2f39e8 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:55.305423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612895c8-9eb6-4651-b501-00bf3f2f39e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:55.305694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612895c8-9eb6-4651-b501-00bf3f2f39e8 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:55.306314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612895c8-9eb6-4651-b501-00bf3f2f39e8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:55.306614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612895c8-9eb6-4651-b501-00bf3f2f39e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:55.307968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612895c8-9eb6-4651-b501-00bf3f2f39e8 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:55.312592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 634/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 30 19:19:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:55.318828 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834f5a9-f731-441c-b7ff-226daa63ea13 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:55.415771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834f5a9-f731-441c-b7ff-226daa63ea13 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:55.447411 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d278c2-b79d-494e-9552-dbf977bce562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:55.447999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d278c2-b79d-494e-9552-dbf977bce562 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:55.448275 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d278c2-b79d-494e-9552-dbf977bce562 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:55.448574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d278c2-b79d-494e-9552-dbf977bce562 None admin] PATH_INFO: `/v3/services` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:55.449187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d278c2-b79d-494e-9552-dbf977bce562 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:19:55.449484 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d278c2-b79d-494e-9552-dbf977bce562 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:55.450831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d278c2-b79d-494e-9552-dbf977bce562 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:55.455030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 658/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 30 19:19:55 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:19:57.561345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81174928-738d-4034-8a47-542ec3375abf None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:57.561860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81174928-738d-4034-8a47-542ec3375abf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:57.561860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81174928-738d-4034-8a47-542ec3375abf None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:57.563228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 635/1293] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:19:57.568818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc4cf4-08bd-40ae-96ad-efe60ddc2272 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:57.569220 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc4cf4-08bd-40ae-96ad-efe60ddc2272 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:57.569363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc4cf4-08bd-40ae-96ad-efe60ddc2272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:57.587396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-32fc4cf4-08bd-40ae-96ad-efe60ddc2272 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:19:57.588590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-32fc4cf4-08bd-40ae-96ad-efe60ddc2272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:19:57.615964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:57.651017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 659/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:19:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:19:57.916959 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81174928-738d-4034-8a47-542ec3375abf None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:19:57.946385 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:19:58.013763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81174928-738d-4034-8a47-542ec3375abf None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:19:58.047416 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3f328-d7ac-4d28-ad08-edffb828dc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:19:58.048000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f328-d7ac-4d28-ad08-edffb828dc9e None admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:19:58.048273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f328-d7ac-4d28-ad08-edffb828dc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:19:58.048567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3f328-d7ac-4d28-ad08-edffb828dc9e None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:19:58.049219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3f328-d7ac-4d28-ad08-edffb828dc9e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:19:58.050802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3f328-d7ac-4d28-ad08-edffb828dc9e None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:19:58.064038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 636/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Sep 30 19:19:57 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:20:00.172773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08843324-6291-40b7-baf3-c94685b52128 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:20:00.173308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08843324-6291-40b7-baf3-c94685b52128 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:20:00.173308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08843324-6291-40b7-baf3-c94685b52128 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:20:00.174629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 660/1296] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:20:00.179714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b588ab7-4317-4044-a8ed-f3fad96fc2fe None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:20:00.180120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b588ab7-4317-4044-a8ed-f3fad96fc2fe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:20:00.180266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b588ab7-4317-4044-a8ed-f3fad96fc2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:20:00.197844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4b588ab7-4317-4044-a8ed-f3fad96fc2fe None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:20:00.199006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4b588ab7-4317-4044-a8ed-f3fad96fc2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:20:00.270164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 637/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:20:00 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:20:00.538086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08843324-6291-40b7-baf3-c94685b52128 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:20:00.634892 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08843324-6291-40b7-baf3-c94685b52128 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:20:00.667562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b3521-40b2-4dbd-b979-6fa5fffdbe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:20:00.668174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b3521-40b2-4dbd-b979-6fa5fffdbe26 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:20:00.668473 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b3521-40b2-4dbd-b979-6fa5fffdbe26 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:20:00.668745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b3521-40b2-4dbd-b979-6fa5fffdbe26 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:20:00.669315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b3521-40b2-4dbd-b979-6fa5fffdbe26 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:20:00.670658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b3521-40b2-4dbd-b979-6fa5fffdbe26 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-958b3521-40b2-4dbd-b979-6fa5fffdbe26 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:20:00.679714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:20:00.680925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:20:00.681992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 661/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 30 19:20:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:20:00.686681 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b588ab7-4317-4044-a8ed-f3fad96fc2fe None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:20:00.704064 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:20:00.781166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b588ab7-4317-4044-a8ed-f3fad96fc2fe None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:20:00.813282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd2ea1-2dd7-49f9-b526-7aa6dd9f4b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:20:00.813811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd2ea1-2dd7-49f9-b526-7aa6dd9f4b4c None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:20:00.814083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd2ea1-2dd7-49f9-b526-7aa6dd9f4b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:20:00.814345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd2ea1-2dd7-49f9-b526-7aa6dd9f4b4c None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:20:00.814953 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd2ea1-2dd7-49f9-b526-7aa6dd9f4b4c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:20:00.815243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd2ea1-2dd7-49f9-b526-7aa6dd9f4b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:20:00.816609 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd2ea1-2dd7-49f9-b526-7aa6dd9f4b4c None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:20:00.821439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 638/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 30 19:20:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:20:00.827221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b3521-40b2-4dbd-b979-6fa5fffdbe26 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:20:00.858500 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:20:00.923733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b3521-40b2-4dbd-b979-6fa5fffdbe26 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:20:00.954668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a82117-d639-4244-91ab-8e2ee58a4209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:20:00.955247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82117-d639-4244-91ab-8e2ee58a4209 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:20:00.955520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82117-d639-4244-91ab-8e2ee58a4209 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:20:00.955791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82117-d639-4244-91ab-8e2ee58a4209 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:20:00.957006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a82117-d639-4244-91ab-8e2ee58a4209 None admin] RBAC: Adding query filter params (interface=public, service_id=68ddc38762974ca1aab5c52af8c0b1b4, region_id=RegionOne) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:20:00.957304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a82117-d639-4244-91ab-8e2ee58a4209 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:20:00.958637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a82117-d639-4244-91ab-8e2ee58a4209 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:20:00.963342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 662/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Sep 30 19:20:00 2022] GET /identity/v3/endpoints?service_id=68ddc38762974ca1aab5c52af8c0b1b4&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:20:00.968772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd2ea1-2dd7-49f9-b526-7aa6dd9f4b4c None admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:20:01.046483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:20:01.064720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd2ea1-2dd7-49f9-b526-7aa6dd9f4b4c None admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:20:01.096579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd6981-a553-4787-ac90-5119cdc65057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:20:01.097697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd6981-a553-4787-ac90-5119cdc65057 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:20:01.097971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd6981-a553-4787-ac90-5119cdc65057 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:20:01.098236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd6981-a553-4787-ac90-5119cdc65057 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:20:01.098823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd6981-a553-4787-ac90-5119cdc65057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:20:01.100166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd6981-a553-4787-ac90-5119cdc65057 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:20:01.106705 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 639/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Sep 30 19:20:00 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:20:03.213052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec8cf0-d4e1-4e09-8314-0c500d796fff None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:20:03.213588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec8cf0-d4e1-4e09-8314-0c500d796fff None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:20:03.213588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec8cf0-d4e1-4e09-8314-0c500d796fff None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:20:03.214921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 663/1302] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:20:03.220291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46dca-516d-4231-994f-980e6d057a23 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:20:03.220753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46dca-516d-4231-994f-980e6d057a23 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:20:03.220856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46dca-516d-4231-994f-980e6d057a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:20:03.239005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d1f46dca-516d-4231-994f-980e6d057a23 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:20:03.240163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d1f46dca-516d-4231-994f-980e6d057a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:20:03.303600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 640/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:20:03 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 19:20:03.568809 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec8cf0-d4e1-4e09-8314-0c500d796fff None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:20:03.660247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:20:03.667534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec8cf0-d4e1-4e09-8314-0c500d796fff None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:20:03.699295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b47ce-8067-4b35-964a-d52b0645f701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:20:03.699906 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b47ce-8067-4b35-964a-d52b0645f701 None admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:20:03.700182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b47ce-8067-4b35-964a-d52b0645f701 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:20:03.700485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b47ce-8067-4b35-964a-d52b0645f701 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:20:03.701087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b47ce-8067-4b35-964a-d52b0645f701 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:20:03.702405 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b47ce-8067-4b35-964a-d52b0645f701 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application [None req-0d5b47ce-8067-4b35-964a-d52b0645f701 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 19:20:03.711413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 19:20:03.712612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application with Lock( Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self._enter() Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application return self.creator() Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application created_value = creator( Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: ERROR keystone.server.flask.application Sep 30 19:20:03.713641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 664/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Sep 30 19:20:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 19:20:03.718312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f46dca-516d-4231-994f-980e6d057a23 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:20:03.790460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:20:03.812358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f46dca-516d-4231-994f-980e6d057a23 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:20:03.844402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc41004-1f92-4649-bb44-a1bf56546d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:20:03.844941 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc41004-1f92-4649-bb44-a1bf56546d43 None admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:20:03.845215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc41004-1f92-4649-bb44-a1bf56546d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:20:03.845476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc41004-1f92-4649-bb44-a1bf56546d43 None admin] PATH_INFO: `/v3/services` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:20:03.846089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc41004-1f92-4649-bb44-a1bf56546d43 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:20:03.846382 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc41004-1f92-4649-bb44-a1bf56546d43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:20:03.847745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc41004-1f92-4649-bb44-a1bf56546d43 None admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:20:03.852620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 641/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Sep 30 19:20:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:20:03.858135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b47ce-8067-4b35-964a-d52b0645f701 None admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:20:03.952553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b47ce-8067-4b35-964a-d52b0645f701 None admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:20:03.984386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2790289-62ad-4348-97b3-a70234449ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:20:03.984992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2790289-62ad-4348-97b3-a70234449ad1 None admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:20:03.985269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2790289-62ad-4348-97b3-a70234449ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:20:03.985537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2790289-62ad-4348-97b3-a70234449ad1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:20:03.986188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2790289-62ad-4348-97b3-a70234449ad1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:20:03.988043 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2790289-62ad-4348-97b3-a70234449ad1 None admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:20:04.006793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 665/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:20:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:20:13.849756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=30 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:20:14.004054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:07.602958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699825a-18da-42a8-bd57-83a33ac3bc00 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:07.603486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699825a-18da-42a8-bd57-83a33ac3bc00 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:07.603486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699825a-18da-42a8-bd57-83a33ac3bc00 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:07.610581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2851f6-ad98-483a-a8c3-ea29afadbac3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:07.611592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2851f6-ad98-483a-a8c3-ea29afadbac3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:07.611703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2851f6-ad98-483a-a8c3-ea29afadbac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:07.616648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 642/1307] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:07 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:07.711935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ad2851f6-ad98-483a-a8c3-ea29afadbac3 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:07.713372 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ad2851f6-ad98-483a-a8c3-ea29afadbac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:07.826637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 666/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:08.119936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b012cf-c66d-4c7c-8c1e-b59818b6cf54 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:08.120214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b012cf-c66d-4c7c-8c1e-b59818b6cf54 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:08.120724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b012cf-c66d-4c7c-8c1e-b59818b6cf54 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:08.122549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 643/1309] 10.222.0.17 () {54 vars in 964 bytes} [Fri Sep 30 19:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:08.128580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b791b80-e522-4afd-8c0e-52f149d2ea59 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:08.129015 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b791b80-e522-4afd-8c0e-52f149d2ea59 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:08.129429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b791b80-e522-4afd-8c0e-52f149d2ea59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:08.217168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3b791b80-e522-4afd-8c0e-52f149d2ea59 None None] MFA Rules not processed for user `18ea49d1f6c040f18511535b7e51a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:08.218363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3b791b80-e522-4afd-8c0e-52f149d2ea59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:08.240696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:08.299301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 667/1310] 10.222.0.17 () {58 vars in 1061 bytes} [Fri Sep 30 19:21:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:08.308165 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b012cf-c66d-4c7c-8c1e-b59818b6cf54 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:08.309909 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b3b012cf-c66d-4c7c-8c1e-b59818b6cf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:08.425170 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b012cf-c66d-4c7c-8c1e-b59818b6cf54 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:08.508838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb3062-aee6-4b89-a36c-e3b344571e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:08.509454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb3062-aee6-4b89-a36c-e3b344571e40 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:08.509713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb3062-aee6-4b89-a36c-e3b344571e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:08.509975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb3062-aee6-4b89-a36c-e3b344571e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:08.511565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-95cb3062-aee6-4b89-a36c-e3b344571e40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:08.567114 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:08.630572 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb3062-aee6-4b89-a36c-e3b344571e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:08.632638 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb3062-aee6-4b89-a36c-e3b344571e40 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:08.657348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 644/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:11.149268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd23206-7907-4640-89d3-d34e09587508 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:11.149858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd23206-7907-4640-89d3-d34e09587508 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:11.149858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd23206-7907-4640-89d3-d34e09587508 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:11.151176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 668/1312] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:11.156406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f1d61-2c84-449d-8f85-c2aa947f5fbc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:11.156838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f1d61-2c84-449d-8f85-c2aa947f5fbc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:11.156941 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f1d61-2c84-449d-8f85-c2aa947f5fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:11.226607 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-787f1d61-2c84-449d-8f85-c2aa947f5fbc None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:11.227780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-787f1d61-2c84-449d-8f85-c2aa947f5fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:11.321961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 645/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:11.613216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd23206-7907-4640-89d3-d34e09587508 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:11.614745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6dd23206-7907-4640-89d3-d34e09587508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:11.715802 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd23206-7907-4640-89d3-d34e09587508 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:11.785934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:11.788553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae86cca-b457-4d1e-a1e3-70139fdd804e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:11.789164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae86cca-b457-4d1e-a1e3-70139fdd804e service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:11.789910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae86cca-b457-4d1e-a1e3-70139fdd804e service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:11.790233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae86cca-b457-4d1e-a1e3-70139fdd804e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:11.792192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0ae86cca-b457-4d1e-a1e3-70139fdd804e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:11.896555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae86cca-b457-4d1e-a1e3-70139fdd804e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:11.897907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae86cca-b457-4d1e-a1e3-70139fdd804e service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:11.916236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 669/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:14.354030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024c467-83c5-4693-86be-6ad2ed8562b8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:14.354643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024c467-83c5-4693-86be-6ad2ed8562b8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:14.354751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024c467-83c5-4693-86be-6ad2ed8562b8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:14.356119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 646/1315] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:14.362200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85bd15-7328-437f-82ff-c55a17d40625 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:14.362668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85bd15-7328-437f-82ff-c55a17d40625 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:14.362769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85bd15-7328-437f-82ff-c55a17d40625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:14.428574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8a85bd15-7328-437f-82ff-c55a17d40625 None None] MFA Rules not processed for user `18593a37ac42418aab61de7fa0ece2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:14.429751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8a85bd15-7328-437f-82ff-c55a17d40625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:14.479327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:14.522748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 670/1316] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:14.806542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024c467-83c5-4693-86be-6ad2ed8562b8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:14.808105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4024c467-83c5-4693-86be-6ad2ed8562b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:14.820479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:14.910505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024c467-83c5-4693-86be-6ad2ed8562b8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:14.980898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd034f35-4b71-49db-b845-aeeb281da580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:14.981492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd034f35-4b71-49db-b845-aeeb281da580 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:14.981771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd034f35-4b71-49db-b845-aeeb281da580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:14.982034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd034f35-4b71-49db-b845-aeeb281da580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:14.983604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dd034f35-4b71-49db-b845-aeeb281da580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:15.045051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:15.090445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd034f35-4b71-49db-b845-aeeb281da580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:15.091813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd034f35-4b71-49db-b845-aeeb281da580 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:15.110077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 647/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:14 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:17.786156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd5d65-f863-4bf0-a09e-4cf576264713 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:17.786690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd5d65-f863-4bf0-a09e-4cf576264713 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:17.786690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd5d65-f863-4bf0-a09e-4cf576264713 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:17.788017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 671/1318] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:17.793532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3fb31-8388-411b-b9b8-a1d100a6565e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:17.793955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3fb31-8388-411b-b9b8-a1d100a6565e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:17.794050 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3fb31-8388-411b-b9b8-a1d100a6565e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:17.859327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c8b3fb31-8388-411b-b9b8-a1d100a6565e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:17.860531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c8b3fb31-8388-411b-b9b8-a1d100a6565e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:17.953513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 648/1319] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:18.240829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd5d65-f863-4bf0-a09e-4cf576264713 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:18.242379 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-29cd5d65-f863-4bf0-a09e-4cf576264713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:18.343207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd5d65-f863-4bf0-a09e-4cf576264713 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:18.414244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53bb1b-3de2-4585-a6e6-f48556425000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:18.415081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53bb1b-3de2-4585-a6e6-f48556425000 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:18.415352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53bb1b-3de2-4585-a6e6-f48556425000 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:18.415617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53bb1b-3de2-4585-a6e6-f48556425000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:18.417235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bf53bb1b-3de2-4585-a6e6-f48556425000 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:18.424502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:18.522115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53bb1b-3de2-4585-a6e6-f48556425000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:18.523468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53bb1b-3de2-4585-a6e6-f48556425000 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:18.541622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 672/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:21.313311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e65ed-4205-4e4f-b63f-0e9b099b2133 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:21.313917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e65ed-4205-4e4f-b63f-0e9b099b2133 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:21.313917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e65ed-4205-4e4f-b63f-0e9b099b2133 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:21.315236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 649/1321] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:21.320452 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880709fa-7054-4401-89a5-a119bccf1b83 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:21.320895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880709fa-7054-4401-89a5-a119bccf1b83 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:21.321120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880709fa-7054-4401-89a5-a119bccf1b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:21.392675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-880709fa-7054-4401-89a5-a119bccf1b83 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:21.393912 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-880709fa-7054-4401-89a5-a119bccf1b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:21.457937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:21.499870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 673/1322] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:21.799765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eab07f-00bc-4d67-add8-cd39251b3577 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:21.800045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eab07f-00bc-4d67-add8-cd39251b3577 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:21.800470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eab07f-00bc-4d67-add8-cd39251b3577 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:21.801806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 650/1323] 10.222.0.17 () {54 vars in 964 bytes} [Fri Sep 30 19:21:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:21.807677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de389d7c-23a2-433d-a311-c2617e4dbd14 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:21.808027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de389d7c-23a2-433d-a311-c2617e4dbd14 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:21.808382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de389d7c-23a2-433d-a311-c2617e4dbd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:21.884069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-de389d7c-23a2-433d-a311-c2617e4dbd14 None None] MFA Rules not processed for user `18ea49d1f6c040f18511535b7e51a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:21.885287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-de389d7c-23a2-433d-a311-c2617e4dbd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:21.959937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 674/1324] 10.222.0.17 () {58 vars in 1061 bytes} [Fri Sep 30 19:21:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:21:21.969024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eab07f-00bc-4d67-add8-cd39251b3577 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:21.970614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-43eab07f-00bc-4d67-add8-cd39251b3577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:21.991531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:22.075682 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eab07f-00bc-4d67-add8-cd39251b3577 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:22.149755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099415b7-4fc2-45eb-b073-dc175405d094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:22.150353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099415b7-4fc2-45eb-b073-dc175405d094 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:22.150626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099415b7-4fc2-45eb-b073-dc175405d094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:22.150891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099415b7-4fc2-45eb-b073-dc175405d094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:22.152471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-099415b7-4fc2-45eb-b073-dc175405d094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:22.223359 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:22.261438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099415b7-4fc2-45eb-b073-dc175405d094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:22.262886 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099415b7-4fc2-45eb-b073-dc175405d094 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:22.280353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 651/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:24.684996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cde18-cdd1-406d-b5e8-12e95fc53b91 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:24.685520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cde18-cdd1-406d-b5e8-12e95fc53b91 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:24.685612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cde18-cdd1-406d-b5e8-12e95fc53b91 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:24.686889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 675/1326] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:24.692236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376aff2-c5b3-4f4e-a812-f5875728ec89 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:24.692671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376aff2-c5b3-4f4e-a812-f5875728ec89 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:24.692812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376aff2-c5b3-4f4e-a812-f5875728ec89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:24.758739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c376aff2-c5b3-4f4e-a812-f5875728ec89 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:24.759841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c376aff2-c5b3-4f4e-a812-f5875728ec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:24.855842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 652/1327] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:21:25.138198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cde18-cdd1-406d-b5e8-12e95fc53b91 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:25.139719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b66cde18-cdd1-406d-b5e8-12e95fc53b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:25.201026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:25.244527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cde18-cdd1-406d-b5e8-12e95fc53b91 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:25.317065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc175d1a-833d-4e2f-b2d5-1795637d2011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:25.317671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc175d1a-833d-4e2f-b2d5-1795637d2011 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:25.317944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc175d1a-833d-4e2f-b2d5-1795637d2011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:25.318210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc175d1a-833d-4e2f-b2d5-1795637d2011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:25.319762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bc175d1a-833d-4e2f-b2d5-1795637d2011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:25.429838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:25.432642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc175d1a-833d-4e2f-b2d5-1795637d2011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:25.434722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc175d1a-833d-4e2f-b2d5-1795637d2011 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:25.453424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 676/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:27.688714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257baf6c-dab3-44be-a2bc-5e25d8a492ef None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:27.689245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257baf6c-dab3-44be-a2bc-5e25d8a492ef None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:27.689245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257baf6c-dab3-44be-a2bc-5e25d8a492ef None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:27.690586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 653/1329] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:27.695983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9e1e3-92d7-437c-9dab-7c130585b45a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:27.696470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9e1e3-92d7-437c-9dab-7c130585b45a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:27.696698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9e1e3-92d7-437c-9dab-7c130585b45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:27.766347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9bd9e1e3-92d7-437c-9dab-7c130585b45a None None] MFA Rules not processed for user `18593a37ac42418aab61de7fa0ece2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:27.768065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9bd9e1e3-92d7-437c-9dab-7c130585b45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:27.865228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 677/1330] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:28.147174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257baf6c-dab3-44be-a2bc-5e25d8a492ef None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:28.148766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-257baf6c-dab3-44be-a2bc-5e25d8a492ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:28.179587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:28.252968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257baf6c-dab3-44be-a2bc-5e25d8a492ef None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:28.326941 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb279ed-ff90-4da2-b2d8-3a7472bb1bcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:28.327540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb279ed-ff90-4da2-b2d8-3a7472bb1bcd service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:28.327813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb279ed-ff90-4da2-b2d8-3a7472bb1bcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:28.328081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb279ed-ff90-4da2-b2d8-3a7472bb1bcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:28.330287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8eb279ed-ff90-4da2-b2d8-3a7472bb1bcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:28.410704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:28.450212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb279ed-ff90-4da2-b2d8-3a7472bb1bcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:28.451801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb279ed-ff90-4da2-b2d8-3a7472bb1bcd service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:28.472744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 654/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:30.854246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ce221-20d8-4f07-9d7f-ec46e3bf2b46 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:30.854808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ce221-20d8-4f07-9d7f-ec46e3bf2b46 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:30.854808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ce221-20d8-4f07-9d7f-ec46e3bf2b46 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:30.856136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 678/1332] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:30.861803 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087fdd7-d5ed-4076-b1c2-7e79f6a72365 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:30.862237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087fdd7-d5ed-4076-b1c2-7e79f6a72365 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:30.862365 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087fdd7-d5ed-4076-b1c2-7e79f6a72365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:30.927339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9087fdd7-d5ed-4076-b1c2-7e79f6a72365 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:30.928563 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9087fdd7-d5ed-4076-b1c2-7e79f6a72365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:31.023361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 655/1333] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:31.309146 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ce221-20d8-4f07-9d7f-ec46e3bf2b46 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:31.310725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9f0ce221-20d8-4f07-9d7f-ec46e3bf2b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:31.378186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:31.413109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ce221-20d8-4f07-9d7f-ec46e3bf2b46 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:31.484868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31f665-69eb-43a1-a232-707b6a63a71b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:31.486014 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31f665-69eb-43a1-a232-707b6a63a71b service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:31.486295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31f665-69eb-43a1-a232-707b6a63a71b service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:31.486566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31f665-69eb-43a1-a232-707b6a63a71b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:31.488165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1b31f665-69eb-43a1-a232-707b6a63a71b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:31.594212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31f665-69eb-43a1-a232-707b6a63a71b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:31.596145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31f665-69eb-43a1-a232-707b6a63a71b service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:31.605937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:31.613898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 679/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:33.820905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b833af1-091e-47b1-8c06-d53b004c20e8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:33.821463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b833af1-091e-47b1-8c06-d53b004c20e8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:33.821463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b833af1-091e-47b1-8c06-d53b004c20e8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:33.822854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 656/1335] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:33.828091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add28ab8-b894-4075-824c-5e41c8ccf3ed None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:33.828569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add28ab8-b894-4075-824c-5e41c8ccf3ed None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:33.828673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add28ab8-b894-4075-824c-5e41c8ccf3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:33.892089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-add28ab8-b894-4075-824c-5e41c8ccf3ed None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:33.893288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-add28ab8-b894-4075-824c-5e41c8ccf3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:33.985847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 680/1336] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:34.269596 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b833af1-091e-47b1-8c06-d53b004c20e8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:34.271141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b833af1-091e-47b1-8c06-d53b004c20e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:34.311676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:34.373614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b833af1-091e-47b1-8c06-d53b004c20e8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:34.446912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc68c87-b7f8-47ae-9ac0-9854de17dfc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:34.447518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68c87-b7f8-47ae-9ac0-9854de17dfc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:34.447793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68c87-b7f8-47ae-9ac0-9854de17dfc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:34.448057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68c87-b7f8-47ae-9ac0-9854de17dfc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:34.449714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9bc68c87-b7f8-47ae-9ac0-9854de17dfc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:34.530769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:34.556546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc68c87-b7f8-47ae-9ac0-9854de17dfc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:34.557983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc68c87-b7f8-47ae-9ac0-9854de17dfc5 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:34.575310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 657/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:37.017056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ad93-9329-4f59-872e-ecafcf89c619 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:37.017590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ad93-9329-4f59-872e-ecafcf89c619 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:37.017590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ad93-9329-4f59-872e-ecafcf89c619 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:37.018985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 681/1338] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:37.024241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633323f-0da2-4026-ac6f-73432b6fa82d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:37.024671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633323f-0da2-4026-ac6f-73432b6fa82d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:37.024851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633323f-0da2-4026-ac6f-73432b6fa82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:37.092018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1633323f-0da2-4026-ac6f-73432b6fa82d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:37.093580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1633323f-0da2-4026-ac6f-73432b6fa82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:37.190132 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 658/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:37.475565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52ad93-9329-4f59-872e-ecafcf89c619 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:37.477196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-de52ad93-9329-4f59-872e-ecafcf89c619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:37.550666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:37.577605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52ad93-9329-4f59-872e-ecafcf89c619 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:37.650887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6d644-88a5-41e3-9401-5288ea011a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:37.651543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6d644-88a5-41e3-9401-5288ea011a96 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:37.651852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6d644-88a5-41e3-9401-5288ea011a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:37.652155 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6d644-88a5-41e3-9401-5288ea011a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:37.653867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-55d6d644-88a5-41e3-9401-5288ea011a96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:37.760740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6d644-88a5-41e3-9401-5288ea011a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:37.762079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6d644-88a5-41e3-9401-5288ea011a96 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:37.779897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 682/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:41.799638 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7fa9b-29ea-461c-a8fa-f712923b50fe None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:41.800217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7fa9b-29ea-461c-a8fa-f712923b50fe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:41.800217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7fa9b-29ea-461c-a8fa-f712923b50fe None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:41.801566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 659/1341] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:41.806940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a313e-db64-4e1a-9f84-a6615eadc1fb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:41.807368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a313e-db64-4e1a-9f84-a6615eadc1fb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:41.807472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a313e-db64-4e1a-9f84-a6615eadc1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:41.812869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:41.874118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-cc6a313e-db64-4e1a-9f84-a6615eadc1fb None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:41.875570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cc6a313e-db64-4e1a-9f84-a6615eadc1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:41.968381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 683/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:42.251457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7fa9b-29ea-461c-a8fa-f712923b50fe None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:42.253029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8bc7fa9b-29ea-461c-a8fa-f712923b50fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:42.302143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:42.356880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7fa9b-29ea-461c-a8fa-f712923b50fe None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:42.437640 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677352cc-3da1-4f1a-8620-d163ce6ec642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:42.438294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677352cc-3da1-4f1a-8620-d163ce6ec642 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:42.438568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677352cc-3da1-4f1a-8620-d163ce6ec642 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:42.438834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677352cc-3da1-4f1a-8620-d163ce6ec642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:42.440450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-677352cc-3da1-4f1a-8620-d163ce6ec642 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:42.530470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:42.548973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677352cc-3da1-4f1a-8620-d163ce6ec642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:42.550410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677352cc-3da1-4f1a-8620-d163ce6ec642 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:42.568785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 660/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:45.368094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ecad-edac-4907-a205-d087f4d37cf5 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:45.368707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ecad-edac-4907-a205-d087f4d37cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:45.368707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ecad-edac-4907-a205-d087f4d37cf5 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:45.370020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 684/1344] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:45.376454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528070-5123-4b4a-ad28-57158f1af053 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:45.376974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528070-5123-4b4a-ad28-57158f1af053 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:45.377126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528070-5123-4b4a-ad28-57158f1af053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:45.443517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b0528070-5123-4b4a-ad28-57158f1af053 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:45.444869 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b0528070-5123-4b4a-ad28-57158f1af053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:45.541944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 661/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:45.828047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5ecad-edac-4907-a205-d087f4d37cf5 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:45.830384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-68d5ecad-edac-4907-a205-d087f4d37cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:45.920902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:45.937233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5ecad-edac-4907-a205-d087f4d37cf5 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:46.007855 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6555235-f176-4f97-8c38-1a560cf2d32f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:46.008488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6555235-f176-4f97-8c38-1a560cf2d32f service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:46.008762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6555235-f176-4f97-8c38-1a560cf2d32f service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:46.009024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6555235-f176-4f97-8c38-1a560cf2d32f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:46.010582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c6555235-f176-4f97-8c38-1a560cf2d32f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:46.131228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6555235-f176-4f97-8c38-1a560cf2d32f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:46.132962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6555235-f176-4f97-8c38-1a560cf2d32f service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:46.151728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 685/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:50.892657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7fe1c-562f-4588-9edd-0147cb1d7805 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:50.893109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7fe1c-562f-4588-9edd-0147cb1d7805 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:50.893209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7fe1c-562f-4588-9edd-0147cb1d7805 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:50.894844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 662/1347] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:21:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:50.900538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d71cfb-eaf9-4457-b0b0-09668fb03939 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:50.901051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d71cfb-eaf9-4457-b0b0-09668fb03939 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:50.901216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d71cfb-eaf9-4457-b0b0-09668fb03939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:50.917270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:50.972878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d3d71cfb-eaf9-4457-b0b0-09668fb03939 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:50.974118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d3d71cfb-eaf9-4457-b0b0-09668fb03939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:51.074831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 686/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:21:51.360732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7fe1c-562f-4588-9edd-0147cb1d7805 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:21:51.362270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e1f7fe1c-562f-4588-9edd-0147cb1d7805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:51.419597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:51.464025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7fe1c-562f-4588-9edd-0147cb1d7805 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:21:51.536816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c72f2c-33c0-4635-a784-351eef0a43b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:21:51.537344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c72f2c-33c0-4635-a784-351eef0a43b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:51.537616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c72f2c-33c0-4635-a784-351eef0a43b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:51.537882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c72f2c-33c0-4635-a784-351eef0a43b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:51.539474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-13c72f2c-33c0-4635-a784-351eef0a43b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:51.644530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:21:51.648059 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c72f2c-33c0-4635-a784-351eef0a43b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:21:51.649707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c72f2c-33c0-4635-a784-351eef0a43b7 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:21:51.668809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 663/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:21:56.575694 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06551ba-c3c6-463b-aac4-ebd382c6a837 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:56.576099 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06551ba-c3c6-463b-aac4-ebd382c6a837 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:56.576210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06551ba-c3c6-463b-aac4-ebd382c6a837 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:56.577845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 687/1350] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:21:56.585742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b4e7-48c6-4a73-a8f0-55d781787da3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:21:56.586477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b4e7-48c6-4a73-a8f0-55d781787da3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:21:56.586898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b4e7-48c6-4a73-a8f0-55d781787da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:21:56.711757 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b065b4e7-48c6-4a73-a8f0-55d781787da3 None None] MFA Rules not processed for user `0a50fd417ae94859bb713b8b2066a9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:21:56.713264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b065b4e7-48c6-4a73-a8f0-55d781787da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:21:56.855905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 664/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:21:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:00.940194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aad80d-236b-46a9-9dc1-e8cf8f591aff None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:00.940846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aad80d-236b-46a9-9dc1-e8cf8f591aff None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:00.940846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aad80d-236b-46a9-9dc1-e8cf8f591aff None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:00.942277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 688/1352] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:00.947155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de22945-1303-4d6d-a53c-cac6fed31191 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:00.947594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de22945-1303-4d6d-a53c-cac6fed31191 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:00.947716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de22945-1303-4d6d-a53c-cac6fed31191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:01.004958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:01.029037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5de22945-1303-4d6d-a53c-cac6fed31191 None None] MFA Rules not processed for user `0a50fd417ae94859bb713b8b2066a9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:01.030352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5de22945-1303-4d6d-a53c-cac6fed31191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:01.073234 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=59 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:01.146584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 665/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:22:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:02.672532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993c4a4-5a27-417c-ac3e-e628082a2836 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:02.673048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993c4a4-5a27-417c-ac3e-e628082a2836 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:02.673048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993c4a4-5a27-417c-ac3e-e628082a2836 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:02.674356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 689/1354] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:02.679487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405771-126b-40fd-af76-080293758b1f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:02.679895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405771-126b-40fd-af76-080293758b1f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:02.680176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405771-126b-40fd-af76-080293758b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:02.757738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4e405771-126b-40fd-af76-080293758b1f None None] MFA Rules not processed for user `0a50fd417ae94859bb713b8b2066a9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:02.758934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4e405771-126b-40fd-af76-080293758b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:02.801246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:02.864909 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 666/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:22:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:22:06.332866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a7adc-9425-4f58-a2c0-5cadd55a5e73 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:06.334763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a7adc-9425-4f58-a2c0-5cadd55a5e73 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:06.335294 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a7adc-9425-4f58-a2c0-5cadd55a5e73 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:06.340444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 690/1356] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:06.342535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578fd98-7691-4fae-a8fb-8c3aed391d60 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:06.342990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578fd98-7691-4fae-a8fb-8c3aed391d60 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:06.343083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578fd98-7691-4fae-a8fb-8c3aed391d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:06.414539 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9578fd98-7691-4fae-a8fb-8c3aed391d60 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:06.415976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9578fd98-7691-4fae-a8fb-8c3aed391d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:06.520577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 667/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:06.809957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45b369-b16e-4c1a-b4f3-c9e08adfefbc None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:06.810719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45b369-b16e-4c1a-b4f3-c9e08adfefbc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:06.811220 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45b369-b16e-4c1a-b4f3-c9e08adfefbc None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:06.813367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 691/1358] 10.222.0.17 () {54 vars in 961 bytes} [Fri Sep 30 19:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:06.823047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20837-eb07-43b3-9576-280b6a4286d3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:06.823871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20837-eb07-43b3-9576-280b6a4286d3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:06.824763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20837-eb07-43b3-9576-280b6a4286d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:06.848312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:06.913509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5ed20837-eb07-43b3-9576-280b6a4286d3 None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:06.915253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5ed20837-eb07-43b3-9576-280b6a4286d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:07.035034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 668/1359] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Sep 30 19:22:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:22:07.043036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45b369-b16e-4c1a-b4f3-c9e08adfefbc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:07.044589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6a45b369-b16e-4c1a-b4f3-c9e08adfefbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:07.198529 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45b369-b16e-4c1a-b4f3-c9e08adfefbc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:07.274626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:07.316462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ef7e2-556c-4ae1-b1f3-ed174dac28ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:07.317116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ef7e2-556c-4ae1-b1f3-ed174dac28ce service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:07.317502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ef7e2-556c-4ae1-b1f3-ed174dac28ce service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:07.317845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ef7e2-556c-4ae1-b1f3-ed174dac28ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:07.319513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4f2ef7e2-556c-4ae1-b1f3-ed174dac28ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:07.423011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ef7e2-556c-4ae1-b1f3-ed174dac28ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:07.424696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ef7e2-556c-4ae1-b1f3-ed174dac28ce service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:07.444160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 692/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:22:09.808860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a01fa-782a-4d03-8d80-372cc16f7a3c None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:09.809378 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a01fa-782a-4d03-8d80-372cc16f7a3c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:09.809378 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a01fa-782a-4d03-8d80-372cc16f7a3c None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:09.810869 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 669/1361] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:09.816229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ee2fa-447a-415b-a092-fa62fdc300d6 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:09.816687 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ee2fa-447a-415b-a092-fa62fdc300d6 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:09.816876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ee2fa-447a-415b-a092-fa62fdc300d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:09.886416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f84ee2fa-447a-415b-a092-fa62fdc300d6 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:09.886416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f84ee2fa-447a-415b-a092-fa62fdc300d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:09.928318 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:09.995678 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 693/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:10.288479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a01fa-782a-4d03-8d80-372cc16f7a3c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:10.291066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f10a01fa-782a-4d03-8d80-372cc16f7a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:10.371905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:10.482393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a01fa-782a-4d03-8d80-372cc16f7a3c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:10.630071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464069c-e181-477e-a1e5-89e5a15df3c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:10.633514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464069c-e181-477e-a1e5-89e5a15df3c5 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:10.633514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464069c-e181-477e-a1e5-89e5a15df3c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:10.633514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464069c-e181-477e-a1e5-89e5a15df3c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:10.634090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8464069c-e181-477e-a1e5-89e5a15df3c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:10.658733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:10.728917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464069c-e181-477e-a1e5-89e5a15df3c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:10.730441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464069c-e181-477e-a1e5-89e5a15df3c5 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:10.749154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 670/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:22:13.223329 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33702c7c-3533-4558-b6ca-8cc8e9001c4f None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:13.223949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33702c7c-3533-4558-b6ca-8cc8e9001c4f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:13.223949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33702c7c-3533-4558-b6ca-8cc8e9001c4f None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:13.225262 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 694/1364] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:13.230769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215869-34a2-42ab-8074-65a487378b8f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:13.231187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215869-34a2-42ab-8074-65a487378b8f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:13.231356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215869-34a2-42ab-8074-65a487378b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:13.299870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-19215869-34a2-42ab-8074-65a487378b8f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:13.301129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-19215869-34a2-42ab-8074-65a487378b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:13.406767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:13.412480 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 671/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:13.699465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611b2b4-6088-4b6d-8bf6-4d80d02b6a9b None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:13.699884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611b2b4-6088-4b6d-8bf6-4d80d02b6a9b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:13.700294 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611b2b4-6088-4b6d-8bf6-4d80d02b6a9b None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:13.701854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 695/1366] 10.222.0.17 () {54 vars in 961 bytes} [Fri Sep 30 19:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:13.707940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aacd02-e051-4d82-ac70-6699dab0110f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:13.708232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aacd02-e051-4d82-ac70-6699dab0110f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:13.708536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aacd02-e051-4d82-ac70-6699dab0110f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:13.786775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-34aacd02-e051-4d82-ac70-6699dab0110f None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:13.787962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-34aacd02-e051-4d82-ac70-6699dab0110f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:13.907409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 672/1367] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Sep 30 19:22:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:13.915402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611b2b4-6088-4b6d-8bf6-4d80d02b6a9b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:13.916925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a611b2b4-6088-4b6d-8bf6-4d80d02b6a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:14.068934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611b2b4-6088-4b6d-8bf6-4d80d02b6a9b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:14.089955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:14.188304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a35dc-8fe1-47cb-b595-0321d55ef045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:14.188941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a35dc-8fe1-47cb-b595-0321d55ef045 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:14.189209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a35dc-8fe1-47cb-b595-0321d55ef045 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:14.189475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a35dc-8fe1-47cb-b595-0321d55ef045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:14.191069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-582a35dc-8fe1-47cb-b595-0321d55ef045 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:14.284251 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a35dc-8fe1-47cb-b595-0321d55ef045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:14.285898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a35dc-8fe1-47cb-b595-0321d55ef045 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:14.305035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 696/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:22:14.467369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76c486-a547-4e04-8003-21462497ecb1 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:14.467946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76c486-a547-4e04-8003-21462497ecb1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:14.467946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76c486-a547-4e04-8003-21462497ecb1 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:14.469539 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 673/1369] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:14.475240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070561e-ea0f-4191-b311-10bde941ad4c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:14.475572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070561e-ea0f-4191-b311-10bde941ad4c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:14.475787 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070561e-ea0f-4191-b311-10bde941ad4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:14.510908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:14.552098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2070561e-ea0f-4191-b311-10bde941ad4c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:14.553501 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2070561e-ea0f-4191-b311-10bde941ad4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:14.665174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 697/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:22:15.004912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fece7e-9643-4fb9-8038-cb400dcf1921 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:15.004912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fece7e-9643-4fb9-8038-cb400dcf1921 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:15.004912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fece7e-9643-4fb9-8038-cb400dcf1921 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:15.004912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 674/1371] 10.222.0.17 () {54 vars in 963 bytes} [Fri Sep 30 19:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:15.008436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72ece6-9464-4e98-9bd5-a5ca0c9ba10a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:15.008724 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72ece6-9464-4e98-9bd5-a5ca0c9ba10a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:15.009000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72ece6-9464-4e98-9bd5-a5ca0c9ba10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:15.088887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1a72ece6-9464-4e98-9bd5-a5ca0c9ba10a None None] MFA Rules not processed for user `65d3020e1f6c414caabbaa185fb7ed48`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:15.090064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1a72ece6-9464-4e98-9bd5-a5ca0c9ba10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:15.139399 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:15.176929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 698/1372] 10.222.0.17 () {58 vars in 1060 bytes} [Fri Sep 30 19:22:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:15.184539 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fece7e-9643-4fb9-8038-cb400dcf1921 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:15.186046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-09fece7e-9643-4fb9-8038-cb400dcf1921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:15.239312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:15.302296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fece7e-9643-4fb9-8038-cb400dcf1921 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:15.382788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af405a92-e37e-442f-9a1a-b259ca9e8bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:15.383799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af405a92-e37e-442f-9a1a-b259ca9e8bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:15.384071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af405a92-e37e-442f-9a1a-b259ca9e8bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:15.384338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af405a92-e37e-442f-9a1a-b259ca9e8bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:15.385988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-af405a92-e37e-442f-9a1a-b259ca9e8bb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:15.507338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:15.524878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af405a92-e37e-442f-9a1a-b259ca9e8bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:15.527506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af405a92-e37e-442f-9a1a-b259ca9e8bb7 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:15.556296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 675/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:22:16.550402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5604134-c133-42d6-bb71-bf41b0ba4069 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:16.550971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5604134-c133-42d6-bb71-bf41b0ba4069 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:16.550971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5604134-c133-42d6-bb71-bf41b0ba4069 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:16.552362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 699/1374] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:16.557832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46486898-a92d-4425-8d04-cfdb9e896a7b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:16.558290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46486898-a92d-4425-8d04-cfdb9e896a7b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:16.558561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46486898-a92d-4425-8d04-cfdb9e896a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:16.625990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-46486898-a92d-4425-8d04-cfdb9e896a7b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:16.627194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46486898-a92d-4425-8d04-cfdb9e896a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:16.728865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 676/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:17.024814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5604134-c133-42d6-bb71-bf41b0ba4069 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:17.026394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a5604134-c133-42d6-bb71-bf41b0ba4069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:17.190206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5604134-c133-42d6-bb71-bf41b0ba4069 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:17.246937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:17.311436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde4cc9-4942-46fc-b5f7-a455dbeaf830 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:17.312113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde4cc9-4942-46fc-b5f7-a455dbeaf830 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:17.312458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde4cc9-4942-46fc-b5f7-a455dbeaf830 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:17.312727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde4cc9-4942-46fc-b5f7-a455dbeaf830 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:17.314332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3dde4cc9-4942-46fc-b5f7-a455dbeaf830 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:17.416979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde4cc9-4942-46fc-b5f7-a455dbeaf830 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:17.418570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde4cc9-4942-46fc-b5f7-a455dbeaf830 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:17.436504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 700/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:22:19.646801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3653-bfe7-40f3-b59e-86e9cfb61ba7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:19.647752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3653-bfe7-40f3-b59e-86e9cfb61ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:19.648173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3653-bfe7-40f3-b59e-86e9cfb61ba7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:19.649881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 677/1377] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:19.655304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ed6f6-eb00-44a7-9fbd-b78f959f6625 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:19.656413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ed6f6-eb00-44a7-9fbd-b78f959f6625 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:19.656600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ed6f6-eb00-44a7-9fbd-b78f959f6625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:19.723219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7b5ed6f6-eb00-44a7-9fbd-b78f959f6625 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:19.724597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7b5ed6f6-eb00-44a7-9fbd-b78f959f6625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:19.731394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:19.836452 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 701/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:20.115138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca3653-bfe7-40f3-b59e-86e9cfb61ba7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:20.116851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-66ca3653-bfe7-40f3-b59e-86e9cfb61ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:20.182952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:20.271849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca3653-bfe7-40f3-b59e-86e9cfb61ba7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:20.394287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97936493-87bb-42bc-a685-805a2f7a8e30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:20.394814 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97936493-87bb-42bc-a685-805a2f7a8e30 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:20.395086 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97936493-87bb-42bc-a685-805a2f7a8e30 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:20.395348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97936493-87bb-42bc-a685-805a2f7a8e30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:20.400003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-97936493-87bb-42bc-a685-805a2f7a8e30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:20.418624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:20.492868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97936493-87bb-42bc-a685-805a2f7a8e30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:20.494553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97936493-87bb-42bc-a685-805a2f7a8e30 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:20.513043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 678/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:22:22.744198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2b5d0-7529-4d6a-8724-b61c1357c5ce None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:22.744778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2b5d0-7529-4d6a-8724-b61c1357c5ce None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:22.745032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2b5d0-7529-4d6a-8724-b61c1357c5ce None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:22.746812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 702/1380] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:22.751880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0036dba-d4d3-46fb-8e58-653f709d580d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:22.752289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0036dba-d4d3-46fb-8e58-653f709d580d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:22.752509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0036dba-d4d3-46fb-8e58-653f709d580d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:22.822260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c0036dba-d4d3-46fb-8e58-653f709d580d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:22.823595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c0036dba-d4d3-46fb-8e58-653f709d580d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:22.976626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:23.001727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 679/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:23.319010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2b5d0-7529-4d6a-8724-b61c1357c5ce None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:23.320578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f5f2b5d0-7529-4d6a-8724-b61c1357c5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:23.483878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:23.496823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2b5d0-7529-4d6a-8724-b61c1357c5ce None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:23.614796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46542d9-15e0-4d33-9d8d-0ef9200660b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:23.615317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46542d9-15e0-4d33-9d8d-0ef9200660b4 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:23.615594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46542d9-15e0-4d33-9d8d-0ef9200660b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:23.615858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46542d9-15e0-4d33-9d8d-0ef9200660b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:23.617458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a46542d9-15e0-4d33-9d8d-0ef9200660b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:23.713859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46542d9-15e0-4d33-9d8d-0ef9200660b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:23.715449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46542d9-15e0-4d33-9d8d-0ef9200660b4 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:23.725816 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:23.734973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 703/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:22:26.527306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d813e63-1b54-43a9-8789-eeffdc2e12cd None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:26.527846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d813e63-1b54-43a9-8789-eeffdc2e12cd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:26.527949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d813e63-1b54-43a9-8789-eeffdc2e12cd None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:26.529702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 680/1383] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:26.535931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bf61f-8637-4e70-95c5-0b9c0a385e27 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:26.536528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bf61f-8637-4e70-95c5-0b9c0a385e27 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:26.536528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bf61f-8637-4e70-95c5-0b9c0a385e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:26.603307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-958bf61f-8637-4e70-95c5-0b9c0a385e27 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:26.604543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-958bf61f-8637-4e70-95c5-0b9c0a385e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:26.704770 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 704/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:26.991337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d813e63-1b54-43a9-8789-eeffdc2e12cd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:26.993356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3d813e63-1b54-43a9-8789-eeffdc2e12cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:27.143671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d813e63-1b54-43a9-8789-eeffdc2e12cd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:27.213409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:27.259811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13aacf3-2363-40da-9e08-677c397d6f37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:27.260259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13aacf3-2363-40da-9e08-677c397d6f37 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:27.260566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13aacf3-2363-40da-9e08-677c397d6f37 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:27.260836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13aacf3-2363-40da-9e08-677c397d6f37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:27.262436 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e13aacf3-2363-40da-9e08-677c397d6f37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:27.374389 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13aacf3-2363-40da-9e08-677c397d6f37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:27.376757 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13aacf3-2363-40da-9e08-677c397d6f37 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:27.399950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 681/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:22:29.562989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bdcfe-bb86-4494-935d-2ad4d8393894 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:29.563578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bdcfe-bb86-4494-935d-2ad4d8393894 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:29.563578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bdcfe-bb86-4494-935d-2ad4d8393894 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:29.565056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 705/1386] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:29.570164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69a067-4c0f-4e7e-bc69-118bf3a4ea2d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:29.570591 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69a067-4c0f-4e7e-bc69-118bf3a4ea2d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:29.570688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69a067-4c0f-4e7e-bc69-118bf3a4ea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:29.635720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-de69a067-4c0f-4e7e-bc69-118bf3a4ea2d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:29.636938 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-de69a067-4c0f-4e7e-bc69-118bf3a4ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:29.670153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:29.734129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 682/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:22:30.021776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bdcfe-bb86-4494-935d-2ad4d8393894 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:30.023292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ae7bdcfe-bb86-4494-935d-2ad4d8393894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:30.102270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:30.171362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bdcfe-bb86-4494-935d-2ad4d8393894 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:30.287955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d953e36-c257-4327-b4b4-3753b14eb865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:30.288514 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d953e36-c257-4327-b4b4-3753b14eb865 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:30.288788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d953e36-c257-4327-b4b4-3753b14eb865 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:30.289307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d953e36-c257-4327-b4b4-3753b14eb865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:30.290882 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6d953e36-c257-4327-b4b4-3753b14eb865 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:30.330037 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:30.382850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d953e36-c257-4327-b4b4-3753b14eb865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:30.384477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d953e36-c257-4327-b4b4-3753b14eb865 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:30.404896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 706/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:22:30.721891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0de91-54e6-486d-bf75-70df3d9f4642 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:30.722490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0de91-54e6-486d-bf75-70df3d9f4642 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:30.722490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0de91-54e6-486d-bf75-70df3d9f4642 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:30.723807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 683/1389] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:30.729097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc77028-3f26-457a-9c32-732a764c2555 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:30.729539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc77028-3f26-457a-9c32-732a764c2555 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:30.729652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc77028-3f26-457a-9c32-732a764c2555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:30.794955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fbc77028-3f26-457a-9c32-732a764c2555 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:30.796702 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fbc77028-3f26-457a-9c32-732a764c2555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:30.891393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 707/1390] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:31.193747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f199a-da9b-4f45-b2f6-3f81b14beaf3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:31.194111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f199a-da9b-4f45-b2f6-3f81b14beaf3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:31.194454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f199a-da9b-4f45-b2f6-3f81b14beaf3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:31.195876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 684/1391] 10.222.0.17 () {54 vars in 962 bytes} [Fri Sep 30 19:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:31.201302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885300c-e6b9-41ca-8522-cfc5093b5a27 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:31.202219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885300c-e6b9-41ca-8522-cfc5093b5a27 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:31.202495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885300c-e6b9-41ca-8522-cfc5093b5a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:31.218083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:31.282739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e885300c-e6b9-41ca-8522-cfc5093b5a27 None None] MFA Rules not processed for user `58a9b371434243daa4176215daefb79c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:31.283858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e885300c-e6b9-41ca-8522-cfc5093b5a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:31.361510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 708/1392] 10.222.0.17 () {58 vars in 1059 bytes} [Fri Sep 30 19:22:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:22:31.370724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f199a-da9b-4f45-b2f6-3f81b14beaf3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:31.372309 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-083f199a-da9b-4f45-b2f6-3f81b14beaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:31.481219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f199a-da9b-4f45-b2f6-3f81b14beaf3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:31.539656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:31.555702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc27050-ac5e-4146-bd5d-90a4b41c1390 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a9b371434243daa4176215daefb79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:31.556324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc27050-ac5e-4146-bd5d-90a4b41c1390 service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:31.556692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc27050-ac5e-4146-bd5d-90a4b41c1390 service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:31.556956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc27050-ac5e-4146-bd5d-90a4b41c1390 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:31.558521 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bfc27050-ac5e-4146-bd5d-90a4b41c1390 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:31.668264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc27050-ac5e-4146-bd5d-90a4b41c1390 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:31.669733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc27050-ac5e-4146-bd5d-90a4b41c1390 service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:31.688929 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 685/1393] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:22:32.440457 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea91e79-1b66-41a6-87e3-3763d7764ba0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:32.440922 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea91e79-1b66-41a6-87e3-3763d7764ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:32.441031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea91e79-1b66-41a6-87e3-3763d7764ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:32.513499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9ea91e79-1b66-41a6-87e3-3763d7764ba0 None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:32.514703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9ea91e79-1b66-41a6-87e3-3763d7764ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:32.569127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:32.593221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1acdc-86a3-40ab-96e8-aa112362d845 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:32.593773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1acdc-86a3-40ab-96e8-aa112362d845 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:32.593773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1acdc-86a3-40ab-96e8-aa112362d845 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:32.595076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 686/1394] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:22:32.600004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32e315-b329-41ef-9dbd-748e963e142a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:32.600279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32e315-b329-41ef-9dbd-748e963e142a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:32.600584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32e315-b329-41ef-9dbd-748e963e142a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:32.601813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 709/1395] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:22:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:32.648392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56ddfb-6077-452c-b7e2-4a3f2017d589 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:32.648879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56ddfb-6077-452c-b7e2-4a3f2017d589 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:32.649158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56ddfb-6077-452c-b7e2-4a3f2017d589 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:32.650505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 710/1396] 10.222.0.17 () {54 vars in 943 bytes} [Fri Sep 30 19:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:22:32.657131 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c20a8-d933-4770-8543-5ffbce744226 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:32.657416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c20a8-d933-4770-8543-5ffbce744226 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:32.657702 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c20a8-d933-4770-8543-5ffbce744226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:32.671566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1c32e315-b329-41ef-9dbd-748e963e142a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:32.672790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1c32e315-b329-41ef-9dbd-748e963e142a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:32.713450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:32.734882 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3f5c20a8-d933-4770-8543-5ffbce744226 None None] MFA Rules not processed for user `b8785c8c5ac84393afcaf88e891f3b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:32.736066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3f5c20a8-d933-4770-8543-5ffbce744226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:32.770813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 687/1397] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:22:32.843581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 711/1398] 10.222.0.17 () {58 vars in 1040 bytes} [Fri Sep 30 19:22:32 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:22:32.853087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32e315-b329-41ef-9dbd-748e963e142a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:32.854364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1c32e315-b329-41ef-9dbd-748e963e142a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:32.956426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32e315-b329-41ef-9dbd-748e963e142a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:33.019473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:33.057186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c20a8-d933-4770-8543-5ffbce744226 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:33.058381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3f5c20a8-d933-4770-8543-5ffbce744226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:33.062342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d06edc-ad28-4ff0-957f-eca2c0a41c5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:33.062957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d06edc-ad28-4ff0-957f-eca2c0a41c5b service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:33.063257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d06edc-ad28-4ff0-957f-eca2c0a41c5b service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:33.063526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d06edc-ad28-4ff0-957f-eca2c0a41c5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:33.065201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-41d06edc-ad28-4ff0-957f-eca2c0a41c5b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:33.088596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:33.152538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d06edc-ad28-4ff0-957f-eca2c0a41c5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:33.154142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d06edc-ad28-4ff0-957f-eca2c0a41c5b service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:33.156136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c20a8-d933-4770-8543-5ffbce744226 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:33.162149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 688/1399] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:22:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:22:33.295904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26a8d6-87ae-41a9-86f5-a999924c0df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:33.296644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26a8d6-87ae-41a9-86f5-a999924c0df0 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:33.296989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26a8d6-87ae-41a9-86f5-a999924c0df0 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:33.297475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26a8d6-87ae-41a9-86f5-a999924c0df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:33.299404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5f26a8d6-87ae-41a9-86f5-a999924c0df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:33.366023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:33.426037 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26a8d6-87ae-41a9-86f5-a999924c0df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:33.427928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26a8d6-87ae-41a9-86f5-a999924c0df0 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:33.455338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 712/1400] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:22:35.667935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dec38-e323-4538-8724-27376921ff85 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:35.668490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dec38-e323-4538-8724-27376921ff85 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:35.668650 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dec38-e323-4538-8724-27376921ff85 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:35.670208 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 689/1401] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:35.675395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc6d25-77b5-4055-b7e6-76ae858a064e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:35.675850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc6d25-77b5-4055-b7e6-76ae858a064e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:35.675970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc6d25-77b5-4055-b7e6-76ae858a064e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:35.747675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-52dc6d25-77b5-4055-b7e6-76ae858a064e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:35.749455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-52dc6d25-77b5-4055-b7e6-76ae858a064e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:35.864532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 713/1402] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:36.225084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8dec38-e323-4538-8724-27376921ff85 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:36.226648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4f8dec38-e323-4538-8724-27376921ff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:36.350812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:36.410373 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8dec38-e323-4538-8724-27376921ff85 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:36.535175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493a5bd-177c-4aa1-b403-698347beb5af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:36.536060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493a5bd-177c-4aa1-b403-698347beb5af service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:36.536335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493a5bd-177c-4aa1-b403-698347beb5af service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:36.536647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493a5bd-177c-4aa1-b403-698347beb5af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:36.538250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6493a5bd-177c-4aa1-b403-698347beb5af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:36.611178 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:36.642342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493a5bd-177c-4aa1-b403-698347beb5af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:36.643930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493a5bd-177c-4aa1-b403-698347beb5af service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:36.663080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 690/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:22:38.874387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fd470-6156-4cc1-9f2e-32edc5fbd301 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:38.875032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fd470-6156-4cc1-9f2e-32edc5fbd301 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:38.875032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fd470-6156-4cc1-9f2e-32edc5fbd301 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:38.876256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 714/1404] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:38.881619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fecf7d-2e2d-43b7-92f7-f7564541c4cd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:38.882076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fecf7d-2e2d-43b7-92f7-f7564541c4cd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:38.882173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fecf7d-2e2d-43b7-92f7-f7564541c4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:38.953963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-46fecf7d-2e2d-43b7-92f7-f7564541c4cd None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:38.955294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46fecf7d-2e2d-43b7-92f7-f7564541c4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:39.064411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 691/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:39.351382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fd470-6156-4cc1-9f2e-32edc5fbd301 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:39.353180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ed6fd470-6156-4cc1-9f2e-32edc5fbd301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:39.375956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:39.529753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fd470-6156-4cc1-9f2e-32edc5fbd301 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:39.630999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:39.656291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36cdc9-2f05-4a89-ac2e-aea3e57f3cfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:39.656846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36cdc9-2f05-4a89-ac2e-aea3e57f3cfc service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:39.657123 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36cdc9-2f05-4a89-ac2e-aea3e57f3cfc service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:39.657391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36cdc9-2f05-4a89-ac2e-aea3e57f3cfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:39.658981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6c36cdc9-2f05-4a89-ac2e-aea3e57f3cfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:39.755665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36cdc9-2f05-4a89-ac2e-aea3e57f3cfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:39.757310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36cdc9-2f05-4a89-ac2e-aea3e57f3cfc service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:39.777434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 715/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:22:41.998346 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e543a86-ab94-42bd-9baa-c7a54dfbff1f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:41.998921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e543a86-ab94-42bd-9baa-c7a54dfbff1f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:41.999043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e543a86-ab94-42bd-9baa-c7a54dfbff1f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:42.001306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 692/1407] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:42.006606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4c3f-22bd-4026-a0fc-f62846900714 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:42.007072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4c3f-22bd-4026-a0fc-f62846900714 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:42.007193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4c3f-22bd-4026-a0fc-f62846900714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:42.081236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fdbf4c3f-22bd-4026-a0fc-f62846900714 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:42.082471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fdbf4c3f-22bd-4026-a0fc-f62846900714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:42.142791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:42.190997 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 716/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:42.484359 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e543a86-ab94-42bd-9baa-c7a54dfbff1f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:42.485902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1e543a86-ab94-42bd-9baa-c7a54dfbff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:42.530851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:42.643586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e543a86-ab94-42bd-9baa-c7a54dfbff1f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:42.770686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e294d6-030c-4b41-8a5f-48b223aedddf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:42.771299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e294d6-030c-4b41-8a5f-48b223aedddf service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:42.771577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e294d6-030c-4b41-8a5f-48b223aedddf service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:42.771846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e294d6-030c-4b41-8a5f-48b223aedddf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:42.773469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-28e294d6-030c-4b41-8a5f-48b223aedddf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:42.775879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:42.874688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e294d6-030c-4b41-8a5f-48b223aedddf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:42.876360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e294d6-030c-4b41-8a5f-48b223aedddf service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:42.897837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 693/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:22:52.189472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=19 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:52.895322 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=44 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:55.860482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a970a-8d23-44e7-9c76-63fe6c8118f5 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:55.861034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a970a-8d23-44e7-9c76-63fe6c8118f5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:55.861034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a970a-8d23-44e7-9c76-63fe6c8118f5 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:55.862367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 717/1410] 10.222.0.17 () {54 vars in 918 bytes} [Fri Sep 30 19:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:55.867655 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d93a6-0e2d-4401-ac74-1479ed78e04a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:55.868125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d93a6-0e2d-4401-ac74-1479ed78e04a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:55.868232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d93a6-0e2d-4401-ac74-1479ed78e04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:55.943439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c57d93a6-0e2d-4401-ac74-1479ed78e04a None None] MFA Rules not processed for user `0a50fd417ae94859bb713b8b2066a9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:55.944758 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c57d93a6-0e2d-4401-ac74-1479ed78e04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:56.048009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 694/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Sep 30 19:22:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:56.062293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756015a-1825-4f86-8efd-91b03a2aa886 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:56.062577 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756015a-1825-4f86-8efd-91b03a2aa886 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:56.062850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756015a-1825-4f86-8efd-91b03a2aa886 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:56.064195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 718/1412] 10.222.0.17 () {54 vars in 918 bytes} [Fri Sep 30 19:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:56.068931 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc62c3-d137-49dd-97f3-f82578c8b3de None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:56.069216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc62c3-d137-49dd-97f3-f82578c8b3de None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:56.069486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc62c3-d137-49dd-97f3-f82578c8b3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:56.141868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:56.144321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-25fc62c3-d137-49dd-97f3-f82578c8b3de None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:56.146049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-25fc62c3-d137-49dd-97f3-f82578c8b3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:56.259010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 695/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Sep 30 19:22:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:58.820475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2296020-c335-42b1-a7a9-d7fb1db25d99 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:58.821021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2296020-c335-42b1-a7a9-d7fb1db25d99 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:58.821021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2296020-c335-42b1-a7a9-d7fb1db25d99 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:58.822335 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:22:58.827512 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba8c57-5e78-4f14-a532-f3c870093f01 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:58.827914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba8c57-5e78-4f14-a532-f3c870093f01 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:58.828058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba8c57-5e78-4f14-a532-f3c870093f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:58.892297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-caba8c57-5e78-4f14-a532-f3c870093f01 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:22:58.893504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-caba8c57-5e78-4f14-a532-f3c870093f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:58.948530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:58.988984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:22:59.274533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2296020-c335-42b1-a7a9-d7fb1db25d99 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:22:59.276078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f2296020-c335-42b1-a7a9-d7fb1db25d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:59.535636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2296020-c335-42b1-a7a9-d7fb1db25d99 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:22:59.608002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:22:59.646630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ea271-fab5-4cbd-aefb-20286ca4e199 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:22:59.647225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ea271-fab5-4cbd-aefb-20286ca4e199 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:22:59.647524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ea271-fab5-4cbd-aefb-20286ca4e199 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:22:59.647787 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ea271-fab5-4cbd-aefb-20286ca4e199 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:22:59.649400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c66ea271-fab5-4cbd-aefb-20286ca4e199 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:22:59.736354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ea271-fab5-4cbd-aefb-20286ca4e199 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:22:59.737951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ea271-fab5-4cbd-aefb-20286ca4e199 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:22:59.755337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 720/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:00.888661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539ee3-47c0-48d5-8811-4ff2be00705b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:00.889225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539ee3-47c0-48d5-8811-4ff2be00705b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:00.889225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539ee3-47c0-48d5-8811-4ff2be00705b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:00.890521 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:00.896632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d84365-9ce7-48bd-81b9-ce0163fb5382 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:00.897072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d84365-9ce7-48bd-81b9-ce0163fb5382 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:00.897171 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d84365-9ce7-48bd-81b9-ce0163fb5382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:00.970050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-25d84365-9ce7-48bd-81b9-ce0163fb5382 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:00.971237 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-25d84365-9ce7-48bd-81b9-ce0163fb5382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:00.997989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:01.066052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:01.213854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f539ee3-47c0-48d5-8811-4ff2be00705b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:01.215383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1f539ee3-47c0-48d5-8811-4ff2be00705b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:01.358497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f539ee3-47c0-48d5-8811-4ff2be00705b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:01.400386 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:01.472297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb59465-9930-4821-bc92-e2a4642321e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:01.472871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59465-9930-4821-bc92-e2a4642321e8 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:01.473146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59465-9930-4821-bc92-e2a4642321e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:01.473415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59465-9930-4821-bc92-e2a4642321e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:01.474991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6eb59465-9930-4821-bc92-e2a4642321e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:01.561491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb59465-9930-4821-bc92-e2a4642321e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:01.563072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb59465-9930-4821-bc92-e2a4642321e8 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:01.580892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 698/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:02.709864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9757c-2a3a-42a5-be4a-4f09f615b116 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:02.710429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9757c-2a3a-42a5-be4a-4f09f615b116 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:02.710429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9757c-2a3a-42a5-be4a-4f09f615b116 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:02.711741 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:02.718772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8895992-4db0-47db-bde8-5a4fd3564173 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:02.719237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8895992-4db0-47db-bde8-5a4fd3564173 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:02.719349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8895992-4db0-47db-bde8-5a4fd3564173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:02.772137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:02.795038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b8895992-4db0-47db-bde8-5a4fd3564173 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:02.796221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b8895992-4db0-47db-bde8-5a4fd3564173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:03.007530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:03.157985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9757c-2a3a-42a5-be4a-4f09f615b116 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:03.159472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-adf9757c-2a3a-42a5-be4a-4f09f615b116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:03.300870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9757c-2a3a-42a5-be4a-4f09f615b116 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:03.313382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:03.411713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f47b40-08e4-46ed-af80-abe3e552bebd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:03.412313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f47b40-08e4-46ed-af80-abe3e552bebd service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:03.412612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f47b40-08e4-46ed-af80-abe3e552bebd service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:03.412879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f47b40-08e4-46ed-af80-abe3e552bebd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:03.414707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-73f47b40-08e4-46ed-af80-abe3e552bebd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:03.499835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f47b40-08e4-46ed-af80-abe3e552bebd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:03.501422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f47b40-08e4-46ed-af80-abe3e552bebd service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:03.518368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 723/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:04.611405 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be3428-6c4c-4926-9223-4791100e5d33 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:04.611950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be3428-6c4c-4926-9223-4791100e5d33 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:04.611950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be3428-6c4c-4926-9223-4791100e5d33 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:04.613196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:04.619260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89b16a-677f-4169-b362-5cff175b67ce None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:04.619692 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89b16a-677f-4169-b362-5cff175b67ce None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:04.619791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89b16a-677f-4169-b362-5cff175b67ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:04.639347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:04.695079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0c89b16a-677f-4169-b362-5cff175b67ce None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:04.696303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0c89b16a-677f-4169-b362-5cff175b67ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:04.792031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:04.940388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be3428-6c4c-4926-9223-4791100e5d33 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:04.941935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-57be3428-6c4c-4926-9223-4791100e5d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:05.049350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:05.084267 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be3428-6c4c-4926-9223-4791100e5d33 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:05.194012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50f446-84fd-4f56-bd47-48e4f2da66db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:05.194535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50f446-84fd-4f56-bd47-48e4f2da66db service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:05.194810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50f446-84fd-4f56-bd47-48e4f2da66db service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:05.195075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50f446-84fd-4f56-bd47-48e4f2da66db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:05.196690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0e50f446-84fd-4f56-bd47-48e4f2da66db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:05.270382 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:05.283597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50f446-84fd-4f56-bd47-48e4f2da66db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:05.285250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50f446-84fd-4f56-bd47-48e4f2da66db service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:05.303333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 701/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:06.399092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acad61f-87dd-4c40-93b1-fa6171fdab9e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:06.399615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acad61f-87dd-4c40-93b1-fa6171fdab9e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:06.399615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acad61f-87dd-4c40-93b1-fa6171fdab9e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:06.400930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:06.407716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5f499-f70b-4cf7-9dd0-3d0a8f6981de None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:06.408200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5f499-f70b-4cf7-9dd0-3d0a8f6981de None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:06.408357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5f499-f70b-4cf7-9dd0-3d0a8f6981de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:06.482943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-00a5f499-f70b-4cf7-9dd0-3d0a8f6981de None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:06.484116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-00a5f499-f70b-4cf7-9dd0-3d0a8f6981de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:06.579697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:06.729637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acad61f-87dd-4c40-93b1-fa6171fdab9e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:06.731731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1acad61f-87dd-4c40-93b1-fa6171fdab9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:06.814944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:06.875253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acad61f-87dd-4c40-93b1-fa6171fdab9e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:06.989383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fa409-d86a-4547-9040-9423cc9596df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:06.989910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fa409-d86a-4547-9040-9423cc9596df service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:06.990186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fa409-d86a-4547-9040-9423cc9596df service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:06.990453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fa409-d86a-4547-9040-9423cc9596df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:06.992036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d22fa409-d86a-4547-9040-9423cc9596df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:07.040510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:07.080776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fa409-d86a-4547-9040-9423cc9596df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:07.082353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fa409-d86a-4547-9040-9423cc9596df service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:07.100391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 726/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:08.140526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0913b-cf67-48b2-9ec2-c02b058fb290 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:08.141136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0913b-cf67-48b2-9ec2-c02b058fb290 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:08.141136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0913b-cf67-48b2-9ec2-c02b058fb290 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:08.142417 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:08.148713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55f26c-af57-4d84-af48-826a2e4b134f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:08.149139 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55f26c-af57-4d84-af48-826a2e4b134f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:08.149248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55f26c-af57-4d84-af48-826a2e4b134f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:08.222766 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5c55f26c-af57-4d84-af48-826a2e4b134f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:08.224111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5c55f26c-af57-4d84-af48-826a2e4b134f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:08.320688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:08.471657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0913b-cf67-48b2-9ec2-c02b058fb290 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:08.473184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-31b0913b-cf67-48b2-9ec2-c02b058fb290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:08.520845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:08.616619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0913b-cf67-48b2-9ec2-c02b058fb290 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:08.726096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e0658-ec78-49f2-b811-5333ce049eb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:08.726694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e0658-ec78-49f2-b811-5333ce049eb0 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:08.726967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e0658-ec78-49f2-b811-5333ce049eb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:08.727233 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e0658-ec78-49f2-b811-5333ce049eb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:08.728835 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b4e0658-ec78-49f2-b811-5333ce049eb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:08.735592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:08.816653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e0658-ec78-49f2-b811-5333ce049eb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:08.818232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e0658-ec78-49f2-b811-5333ce049eb0 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:08.836037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 704/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:09.882186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45650c-bced-4924-a7a8-d9be4209c9a2 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:09.882754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45650c-bced-4924-a7a8-d9be4209c9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:09.882754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45650c-bced-4924-a7a8-d9be4209c9a2 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:09.884061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:09.890256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e620e-7b7d-4f22-b3e3-1e5228ec15af None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:09.890717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e620e-7b7d-4f22-b3e3-1e5228ec15af None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:09.890848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e620e-7b7d-4f22-b3e3-1e5228ec15af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:09.966533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ff3e620e-7b7d-4f22-b3e3-1e5228ec15af None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:09.967751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ff3e620e-7b7d-4f22-b3e3-1e5228ec15af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:10.032800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:10.061999 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:10.210543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45650c-bced-4924-a7a8-d9be4209c9a2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:10.212066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-db45650c-bced-4924-a7a8-d9be4209c9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:10.229628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:10.355266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45650c-bced-4924-a7a8-d9be4209c9a2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:10.440461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:10.465948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eab994-3d2b-40c6-87b3-437ac23e1cb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:10.466480 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eab994-3d2b-40c6-87b3-437ac23e1cb9 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:10.466754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eab994-3d2b-40c6-87b3-437ac23e1cb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:10.467020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eab994-3d2b-40c6-87b3-437ac23e1cb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:10.468627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-26eab994-3d2b-40c6-87b3-437ac23e1cb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:10.555372 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eab994-3d2b-40c6-87b3-437ac23e1cb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:10.556993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eab994-3d2b-40c6-87b3-437ac23e1cb9 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:10.574835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 729/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:11.616976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef5511-caa0-40aa-856f-654284589b4f None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:11.617552 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef5511-caa0-40aa-856f-654284589b4f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:11.617552 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef5511-caa0-40aa-856f-654284589b4f None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:11.618829 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:11.625667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ce1e2-0cb7-4519-97d0-08e434105f1c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:11.626076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ce1e2-0cb7-4519-97d0-08e434105f1c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:11.626214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ce1e2-0cb7-4519-97d0-08e434105f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:11.701620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-032ce1e2-0cb7-4519-97d0-08e434105f1c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:11.702821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-032ce1e2-0cb7-4519-97d0-08e434105f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:11.745049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:11.798996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:11.949975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef5511-caa0-40aa-856f-654284589b4f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:11.951481 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c9ef5511-caa0-40aa-856f-654284589b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:12.097829 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef5511-caa0-40aa-856f-654284589b4f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:12.149916 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:12.207126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb6251-b1c4-4dbb-a49c-4aa907dff655 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:12.207722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb6251-b1c4-4dbb-a49c-4aa907dff655 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:12.207991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb6251-b1c4-4dbb-a49c-4aa907dff655 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:12.208263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb6251-b1c4-4dbb-a49c-4aa907dff655 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:12.209845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e8eb6251-b1c4-4dbb-a49c-4aa907dff655 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:12.296734 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb6251-b1c4-4dbb-a49c-4aa907dff655 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:12.298318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb6251-b1c4-4dbb-a49c-4aa907dff655 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:12.315843 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 707/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:13.367575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ace49-edec-4c5f-8153-afa2c4b16a4b None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:13.368117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ace49-edec-4c5f-8153-afa2c4b16a4b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:13.368117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ace49-edec-4c5f-8153-afa2c4b16a4b None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:13.369524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:13.376799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd1316-4b2d-4ef9-84de-a9029271c5bd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:13.377270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd1316-4b2d-4ef9-84de-a9029271c5bd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:13.377361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd1316-4b2d-4ef9-84de-a9029271c5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:13.452495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b4fd1316-4b2d-4ef9-84de-a9029271c5bd None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:13.453662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b4fd1316-4b2d-4ef9-84de-a9029271c5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:13.455617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:13.546941 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:13.694658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ace49-edec-4c5f-8153-afa2c4b16a4b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:13.696154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e86ace49-edec-4c5f-8153-afa2c4b16a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:13.837394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ace49-edec-4c5f-8153-afa2c4b16a4b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:13.864936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:13.947005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb54926-3029-424e-84b0-0c3a28760e35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:13.947532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb54926-3029-424e-84b0-0c3a28760e35 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:13.947803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb54926-3029-424e-84b0-0c3a28760e35 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:13.948064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb54926-3029-424e-84b0-0c3a28760e35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:13.949699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0fb54926-3029-424e-84b0-0c3a28760e35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:14.035657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb54926-3029-424e-84b0-0c3a28760e35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:14.037259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb54926-3029-424e-84b0-0c3a28760e35 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:14.055313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 732/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:16.096193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b635c60-63e6-4f1c-889f-3c9a8addbae2 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:16.096797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b635c60-63e6-4f1c-889f-3c9a8addbae2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:16.096797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b635c60-63e6-4f1c-889f-3c9a8addbae2 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:16.098088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:16.103308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68237328-8719-4642-8b9b-05a85868e2ca None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:16.103737 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68237328-8719-4642-8b9b-05a85868e2ca None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:16.103875 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68237328-8719-4642-8b9b-05a85868e2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:16.143588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:16.168300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-68237328-8719-4642-8b9b-05a85868e2ca None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:16.169506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-68237328-8719-4642-8b9b-05a85868e2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:16.263747 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:23:16.547556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b635c60-63e6-4f1c-889f-3c9a8addbae2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:16.549103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5b635c60-63e6-4f1c-889f-3c9a8addbae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:16.649652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b635c60-63e6-4f1c-889f-3c9a8addbae2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:16.681646 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:16.725327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ab626-659f-4430-91ba-2e32204b5002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:16.725978 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ab626-659f-4430-91ba-2e32204b5002 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:16.726252 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ab626-659f-4430-91ba-2e32204b5002 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:16.726518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ab626-659f-4430-91ba-2e32204b5002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:16.728169 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a26ab626-659f-4430-91ba-2e32204b5002 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:16.833937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ab626-659f-4430-91ba-2e32204b5002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:16.835360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ab626-659f-4430-91ba-2e32204b5002 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:16.854660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 710/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:23:19.163747 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1f588-6def-44b3-817a-20122bc55971 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:19.164303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1f588-6def-44b3-817a-20122bc55971 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:19.164303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1f588-6def-44b3-817a-20122bc55971 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:19.165679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:19.171525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6be1b-5daf-4fa8-961b-cb8ccb64bd4d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:19.171969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6be1b-5daf-4fa8-961b-cb8ccb64bd4d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:19.172081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6be1b-5daf-4fa8-961b-cb8ccb64bd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:19.235277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-11d6be1b-5daf-4fa8-961b-cb8ccb64bd4d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:19.236478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-11d6be1b-5daf-4fa8-961b-cb8ccb64bd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:19.248746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:19.333405 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:19.615208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1f588-6def-44b3-817a-20122bc55971 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:19.616779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d7f1f588-6def-44b3-817a-20122bc55971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:19.718469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1f588-6def-44b3-817a-20122bc55971 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:19.730960 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:19.791260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17530fef-90b2-4b61-8bfe-b3db028ed597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:19.791862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17530fef-90b2-4b61-8bfe-b3db028ed597 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:19.792138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17530fef-90b2-4b61-8bfe-b3db028ed597 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:19.792434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17530fef-90b2-4b61-8bfe-b3db028ed597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:19.794056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-17530fef-90b2-4b61-8bfe-b3db028ed597 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:19.902137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17530fef-90b2-4b61-8bfe-b3db028ed597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:19.903477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17530fef-90b2-4b61-8bfe-b3db028ed597 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:19.921284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 735/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:22.353729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109a807-2c98-4b95-a1db-4102eacca91b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:22.354279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109a807-2c98-4b95-a1db-4102eacca91b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:22.354279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109a807-2c98-4b95-a1db-4102eacca91b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:22.355688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:22.361034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131924e-32d1-4677-b5ac-83839f99e984 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:22.361446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131924e-32d1-4677-b5ac-83839f99e984 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:22.361576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131924e-32d1-4677-b5ac-83839f99e984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:22.416651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:22.435721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c131924e-32d1-4677-b5ac-83839f99e984 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:22.436947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c131924e-32d1-4677-b5ac-83839f99e984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:22.530305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:22.812912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109a807-2c98-4b95-a1db-4102eacca91b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:22.814454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4109a807-2c98-4b95-a1db-4102eacca91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:22.915485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109a807-2c98-4b95-a1db-4102eacca91b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:22.954497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:22.988238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d39bc-6b6b-445d-9dbc-020ffc4fb1ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:22.988879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d39bc-6b6b-445d-9dbc-020ffc4fb1ea service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:22.989156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d39bc-6b6b-445d-9dbc-020ffc4fb1ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:22.989423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d39bc-6b6b-445d-9dbc-020ffc4fb1ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:22.991004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4c4d39bc-6b6b-445d-9dbc-020ffc4fb1ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:23.097847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d39bc-6b6b-445d-9dbc-020ffc4fb1ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:23.099218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d39bc-6b6b-445d-9dbc-020ffc4fb1ea service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:23.117567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 713/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:25.369278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aea624-45f4-4e0d-b5c1-2b00fcd25f64 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:25.369826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aea624-45f4-4e0d-b5c1-2b00fcd25f64 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:25.369826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aea624-45f4-4e0d-b5c1-2b00fcd25f64 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:25.371108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:25.376487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae16475-01f5-47d0-90c1-73ac0c55374c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:25.376911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae16475-01f5-47d0-90c1-73ac0c55374c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:25.377039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae16475-01f5-47d0-90c1-73ac0c55374c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:25.443123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fae16475-01f5-47d0-90c1-73ac0c55374c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:25.444306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fae16475-01f5-47d0-90c1-73ac0c55374c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:25.464778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:25.539434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:25.820116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aea624-45f4-4e0d-b5c1-2b00fcd25f64 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:25.821682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a5aea624-45f4-4e0d-b5c1-2b00fcd25f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:25.924136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aea624-45f4-4e0d-b5c1-2b00fcd25f64 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:25.943288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:25.995722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5b481-5607-4728-9e28-671986b5d1b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:25.996321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5b481-5607-4728-9e28-671986b5d1b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:25.996619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5b481-5607-4728-9e28-671986b5d1b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:25.996884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5b481-5607-4728-9e28-671986b5d1b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:25.998466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-12d5b481-5607-4728-9e28-671986b5d1b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:26.104490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5b481-5607-4728-9e28-671986b5d1b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:26.106424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5b481-5607-4728-9e28-671986b5d1b9 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:26.123425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 738/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:28.306614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437eb7-fb6f-44ec-b2b3-765ab87672b7 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:28.307198 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437eb7-fb6f-44ec-b2b3-765ab87672b7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:28.307198 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437eb7-fb6f-44ec-b2b3-765ab87672b7 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:28.309043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:28.314281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d3562-c935-4428-bc03-4c7a5e1411e4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:28.314704 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d3562-c935-4428-bc03-4c7a5e1411e4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:28.314821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d3562-c935-4428-bc03-4c7a5e1411e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:28.376229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:28.378857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9a0d3562-c935-4428-bc03-4c7a5e1411e4 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:28.380725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9a0d3562-c935-4428-bc03-4c7a5e1411e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:28.478842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:23:28.765411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc437eb7-fb6f-44ec-b2b3-765ab87672b7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:28.766931 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cc437eb7-fb6f-44ec-b2b3-765ab87672b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:28.867156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc437eb7-fb6f-44ec-b2b3-765ab87672b7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:28.915330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:28.938774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfccecd-b406-4c86-97a9-0162b0dcf599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:28.939364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccecd-b406-4c86-97a9-0162b0dcf599 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:28.939639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccecd-b406-4c86-97a9-0162b0dcf599 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:28.939902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfccecd-b406-4c86-97a9-0162b0dcf599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:28.941491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3dfccecd-b406-4c86-97a9-0162b0dcf599 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:29.047254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfccecd-b406-4c86-97a9-0162b0dcf599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:29.048759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfccecd-b406-4c86-97a9-0162b0dcf599 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:29.066578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 716/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:31.253212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e3b25-c003-4eb6-8c8e-d0230ae4ae1e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:31.253772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e3b25-c003-4eb6-8c8e-d0230ae4ae1e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:31.253772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e3b25-c003-4eb6-8c8e-d0230ae4ae1e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:31.255076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:31.260457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4b609-23fd-42e2-be89-69edf730b727 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:31.261459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4b609-23fd-42e2-be89-69edf730b727 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:31.261597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4b609-23fd-42e2-be89-69edf730b727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:31.327536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f6b4b609-23fd-42e2-be89-69edf730b727 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:31.328678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f6b4b609-23fd-42e2-be89-69edf730b727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:31.356358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:31.425361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:31.707911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e3b25-c003-4eb6-8c8e-d0230ae4ae1e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:31.709500 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-240e3b25-c003-4eb6-8c8e-d0230ae4ae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:31.810242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e3b25-c003-4eb6-8c8e-d0230ae4ae1e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:31.837406 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:31.882445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5990e-e0cc-4e24-a06d-51f3f32f4d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:31.882990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5990e-e0cc-4e24-a06d-51f3f32f4d08 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:31.883261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5990e-e0cc-4e24-a06d-51f3f32f4d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:31.883524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5990e-e0cc-4e24-a06d-51f3f32f4d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:31.885117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-60a5990e-e0cc-4e24-a06d-51f3f32f4d08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:31.990812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5990e-e0cc-4e24-a06d-51f3f32f4d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:31.992165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5990e-e0cc-4e24-a06d-51f3f32f4d08 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:32.009957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 741/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:34.263265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3112b67-62d5-4540-aa6d-0dad4b54f6e0 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:34.263982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3112b67-62d5-4540-aa6d-0dad4b54f6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:34.263982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3112b67-62d5-4540-aa6d-0dad4b54f6e0 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:34.265447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:34.271202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bbda-67c3-46db-bd8b-042e4064de3b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:34.271648 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bbda-67c3-46db-bd8b-042e4064de3b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:34.271749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289bbda-67c3-46db-bd8b-042e4064de3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:34.336748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a289bbda-67c3-46db-bd8b-042e4064de3b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:34.337948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a289bbda-67c3-46db-bd8b-042e4064de3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:34.346362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:34.435006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:34.723231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3112b67-62d5-4540-aa6d-0dad4b54f6e0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:34.724808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f3112b67-62d5-4540-aa6d-0dad4b54f6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:34.826030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3112b67-62d5-4540-aa6d-0dad4b54f6e0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:34.881426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:34.899995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a070f-dc0f-4a19-88ac-e605031c9599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:34.900630 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a070f-dc0f-4a19-88ac-e605031c9599 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:34.900915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a070f-dc0f-4a19-88ac-e605031c9599 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:34.901209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a070f-dc0f-4a19-88ac-e605031c9599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:34.902825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c92a070f-dc0f-4a19-88ac-e605031c9599 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:35.013000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a070f-dc0f-4a19-88ac-e605031c9599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:35.014350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a070f-dc0f-4a19-88ac-e605031c9599 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:35.032587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 719/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:37.215982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70514e5-6b98-4d4b-a7a1-f8949ad7f758 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:37.216604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70514e5-6b98-4d4b-a7a1-f8949ad7f758 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:37.216604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70514e5-6b98-4d4b-a7a1-f8949ad7f758 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:37.217886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:37.223116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c82425-429a-4205-9b85-3e816b487a70 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:37.223505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c82425-429a-4205-9b85-3e816b487a70 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:37.223662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c82425-429a-4205-9b85-3e816b487a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:37.289325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-95c82425-429a-4205-9b85-3e816b487a70 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:37.290437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-95c82425-429a-4205-9b85-3e816b487a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:37.328521 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:37.391164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:37.682864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70514e5-6b98-4d4b-a7a1-f8949ad7f758 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:37.684431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f70514e5-6b98-4d4b-a7a1-f8949ad7f758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:37.786913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70514e5-6b98-4d4b-a7a1-f8949ad7f758 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:37.825322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:37.860468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c8402-bb41-4041-bd06-f57f67cd484a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:37.861071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c8402-bb41-4041-bd06-f57f67cd484a service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:37.861341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c8402-bb41-4041-bd06-f57f67cd484a service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:37.861624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c8402-bb41-4041-bd06-f57f67cd484a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:37.863184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ee2c8402-bb41-4041-bd06-f57f67cd484a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:37.969381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c8402-bb41-4041-bd06-f57f67cd484a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:37.970839 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c8402-bb41-4041-bd06-f57f67cd484a service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:37.988769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 744/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:40.129677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eb58f-e952-4df5-accd-e18c39d230fb None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:40.130257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eb58f-e952-4df5-accd-e18c39d230fb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:40.130257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eb58f-e952-4df5-accd-e18c39d230fb None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:40.131553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:40.137494 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c78fd-37c3-43e7-babc-22f9a27bfda7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:40.137941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c78fd-37c3-43e7-babc-22f9a27bfda7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:40.138054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c78fd-37c3-43e7-babc-22f9a27bfda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:40.201938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d63c78fd-37c3-43e7-babc-22f9a27bfda7 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:40.203112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d63c78fd-37c3-43e7-babc-22f9a27bfda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:40.220531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:40.296922 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:23:40.579273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eb58f-e952-4df5-accd-e18c39d230fb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:40.580916 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eb4eb58f-e952-4df5-accd-e18c39d230fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:40.682485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eb58f-e952-4df5-accd-e18c39d230fb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:40.748005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:40.755145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f35d60-78f6-4baa-90a3-2609066b8f50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:40.755748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f35d60-78f6-4baa-90a3-2609066b8f50 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:40.756024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f35d60-78f6-4baa-90a3-2609066b8f50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:40.756294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f35d60-78f6-4baa-90a3-2609066b8f50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:40.757906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f4f35d60-78f6-4baa-90a3-2609066b8f50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:40.865335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f35d60-78f6-4baa-90a3-2609066b8f50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:40.866761 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f35d60-78f6-4baa-90a3-2609066b8f50 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:40.883982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 722/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:43.030267 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314abd4-7f73-4531-8bb7-884ed02e3ce0 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:43.030699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314abd4-7f73-4531-8bb7-884ed02e3ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:43.030802 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314abd4-7f73-4531-8bb7-884ed02e3ce0 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:43.032167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:23:43.037900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1e8be-2ca0-45b2-9bb3-8ea03d963cbe None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:43.038322 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1e8be-2ca0-45b2-9bb3-8ea03d963cbe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:43.038419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1e8be-2ca0-45b2-9bb3-8ea03d963cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:43.104061 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f5c1e8be-2ca0-45b2-9bb3-8ea03d963cbe None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:23:43.105289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f5c1e8be-2ca0-45b2-9bb3-8ea03d963cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:43.153114 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:43.201278 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:23:43.483802 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314abd4-7f73-4531-8bb7-884ed02e3ce0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:23:43.485346 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0314abd4-7f73-4531-8bb7-884ed02e3ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:43.584592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314abd4-7f73-4531-8bb7-884ed02e3ce0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:23:43.632823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:43.657989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61303187-de48-4b44-9274-5b146f5af55e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:23:43.658556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61303187-de48-4b44-9274-5b146f5af55e service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:23:43.658830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61303187-de48-4b44-9274-5b146f5af55e service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:23:43.659090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61303187-de48-4b44-9274-5b146f5af55e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:23:43.660695 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-61303187-de48-4b44-9274-5b146f5af55e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:23:43.766238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61303187-de48-4b44-9274-5b146f5af55e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:23:43.767689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61303187-de48-4b44-9274-5b146f5af55e service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:23:43.785278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 747/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:23:53.199710 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=22 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:23:53.783095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=66 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:03.607583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34201a72-f99c-4eb3-8723-c56ebad09f94 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:03.608143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34201a72-f99c-4eb3-8723-c56ebad09f94 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:03.608143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34201a72-f99c-4eb3-8723-c56ebad09f94 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:03.609497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 724/1471] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:24:03.614775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d043d-2a0c-42ee-b361-7a3bf7ce0cf6 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:03.615170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d043d-2a0c-42ee-b361-7a3bf7ce0cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:03.615312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d043d-2a0c-42ee-b361-7a3bf7ce0cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:03.679148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-532d043d-2a0c-42ee-b361-7a3bf7ce0cf6 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:24:03.680247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-532d043d-2a0c-42ee-b361-7a3bf7ce0cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:03.773331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 748/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:24:04.068564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023e768-c10e-4489-81eb-1610eb716abe None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:04.068845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023e768-c10e-4489-81eb-1610eb716abe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:04.069113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023e768-c10e-4489-81eb-1610eb716abe None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:04.070449 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 725/1473] 10.222.0.17 () {54 vars in 962 bytes} [Fri Sep 30 19:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:24:04.075825 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f989ac8-869f-40f3-a3a3-5ac765d0fd1e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:04.076106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f989ac8-869f-40f3-a3a3-5ac765d0fd1e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:04.076396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f989ac8-869f-40f3-a3a3-5ac765d0fd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:04.149159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9f989ac8-869f-40f3-a3a3-5ac765d0fd1e None None] MFA Rules not processed for user `58a9b371434243daa4176215daefb79c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:24:04.150353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9f989ac8-869f-40f3-a3a3-5ac765d0fd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:04.170166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:04.223294 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 749/1474] 10.222.0.17 () {58 vars in 1059 bytes} [Fri Sep 30 19:24:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:24:04.231203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023e768-c10e-4489-81eb-1610eb716abe None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:24:04.232736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f023e768-c10e-4489-81eb-1610eb716abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:04.337909 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023e768-c10e-4489-81eb-1610eb716abe None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:24:04.410685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f02dd-2a42-4de0-8b73-64984eaa4e3f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a9b371434243daa4176215daefb79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:24:04.411289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f02dd-2a42-4de0-8b73-64984eaa4e3f service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:04.411561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f02dd-2a42-4de0-8b73-64984eaa4e3f service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:04.411823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f02dd-2a42-4de0-8b73-64984eaa4e3f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:04.413675 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fb6f02dd-2a42-4de0-8b73-64984eaa4e3f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:04.467041 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:04.521109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f02dd-2a42-4de0-8b73-64984eaa4e3f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:24:04.522531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f02dd-2a42-4de0-8b73-64984eaa4e3f service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:24:04.541140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 726/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:24:06.776749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608dbd1-7b47-46a0-9daf-6a9ce88938ec None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:06.777327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608dbd1-7b47-46a0-9daf-6a9ce88938ec None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:06.777327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608dbd1-7b47-46a0-9daf-6a9ce88938ec None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:06.778647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 750/1476] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:24:06.783837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b13e75-13be-488d-9d14-93a7781c0cdc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:06.784220 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b13e75-13be-488d-9d14-93a7781c0cdc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:06.784415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b13e75-13be-488d-9d14-93a7781c0cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:06.850693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-34b13e75-13be-488d-9d14-93a7781c0cdc None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:24:06.851887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-34b13e75-13be-488d-9d14-93a7781c0cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:06.947627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 727/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:24:07.227600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608dbd1-7b47-46a0-9daf-6a9ce88938ec None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:24:07.229740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3608dbd1-7b47-46a0-9daf-6a9ce88938ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:07.351096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608dbd1-7b47-46a0-9daf-6a9ce88938ec None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:24:07.400542 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:07.444015 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219eeeb-a353-4eaa-9ee3-d428969a7c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:24:07.444650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219eeeb-a353-4eaa-9ee3-d428969a7c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:07.444926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219eeeb-a353-4eaa-9ee3-d428969a7c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:07.445194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219eeeb-a353-4eaa-9ee3-d428969a7c36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:07.445821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219eeeb-a353-4eaa-9ee3-d428969a7c36 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:24:07.447556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-4219eeeb-a353-4eaa-9ee3-d428969a7c36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:24:07.447556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 30 19:24:07.447556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: Sep 30 19:24:07.448709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219eeeb-a353-4eaa-9ee3-d428969a7c36 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:24:07.467566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 751/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Sep 30 19:24:07 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:24:09.606095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42585057-dfdc-4e9c-8ad2-7da4f8b8e6bd None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:09.606657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42585057-dfdc-4e9c-8ad2-7da4f8b8e6bd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:09.606657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42585057-dfdc-4e9c-8ad2-7da4f8b8e6bd None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:09.607900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 728/1479] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:24:09.613247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467b9e1-7a38-4958-bc3c-29c9ebe5be56 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:09.613698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467b9e1-7a38-4958-bc3c-29c9ebe5be56 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:09.613819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467b9e1-7a38-4958-bc3c-29c9ebe5be56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:09.677967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9467b9e1-7a38-4958-bc3c-29c9ebe5be56 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:24:09.679153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9467b9e1-7a38-4958-bc3c-29c9ebe5be56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:09.773325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 752/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:24:10.066173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42585057-dfdc-4e9c-8ad2-7da4f8b8e6bd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:24:10.067769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-42585057-dfdc-4e9c-8ad2-7da4f8b8e6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:10.077545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:10.210820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42585057-dfdc-4e9c-8ad2-7da4f8b8e6bd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:24:10.288711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:10.321847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed421564-3af2-47e0-ba26-eca61ac5d774 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:24:10.322344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed421564-3af2-47e0-ba26-eca61ac5d774 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:10.322617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed421564-3af2-47e0-ba26-eca61ac5d774 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:10.322884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed421564-3af2-47e0-ba26-eca61ac5d774 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:10.324475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ed421564-3af2-47e0-ba26-eca61ac5d774 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:10.410533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed421564-3af2-47e0-ba26-eca61ac5d774 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:24:10.412375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed421564-3af2-47e0-ba26-eca61ac5d774 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:24:10.429851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 729/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:24:12.737335 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ac2d-b792-422a-910a-06f1209f3744 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:12.737911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ac2d-b792-422a-910a-06f1209f3744 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:12.737911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0ac2d-b792-422a-910a-06f1209f3744 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:12.739256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 753/1482] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:24:12.744709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318a3a7-22a3-4508-bda1-a3ae8315849a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:12.745137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318a3a7-22a3-4508-bda1-a3ae8315849a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:12.745245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318a3a7-22a3-4508-bda1-a3ae8315849a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:12.812186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9318a3a7-22a3-4508-bda1-a3ae8315849a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:24:12.813406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9318a3a7-22a3-4508-bda1-a3ae8315849a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:12.859440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:12.909325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 730/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:24:13.195516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0ac2d-b792-422a-910a-06f1209f3744 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:24:13.197714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-13f0ac2d-b792-422a-910a-06f1209f3744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:13.220520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:13.298950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0ac2d-b792-422a-910a-06f1209f3744 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:24:13.372710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a574e-9fee-430b-aedb-ab569dbaabba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:24:13.373306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a574e-9fee-430b-aedb-ab569dbaabba service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:13.373576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a574e-9fee-430b-aedb-ab569dbaabba service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:13.373837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a574e-9fee-430b-aedb-ab569dbaabba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:13.375400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d45a574e-9fee-430b-aedb-ab569dbaabba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:13.453058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:13.487411 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a574e-9fee-430b-aedb-ab569dbaabba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:24:13.488853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a574e-9fee-430b-aedb-ab569dbaabba service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:24:13.506530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 754/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:24:15.684743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b6217-5349-4cff-a578-44fbd76baf3e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:15.685293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b6217-5349-4cff-a578-44fbd76baf3e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:15.685293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b6217-5349-4cff-a578-44fbd76baf3e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:15.686642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 731/1485] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:24:15.691899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2025cb4-2740-49ec-8da8-836e38c00ba8 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:15.692316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2025cb4-2740-49ec-8da8-836e38c00ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:15.692461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2025cb4-2740-49ec-8da8-836e38c00ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:15.758123 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d2025cb4-2740-49ec-8da8-836e38c00ba8 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:24:15.759844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d2025cb4-2740-49ec-8da8-836e38c00ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:15.856410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 755/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:24:16.143524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b6217-5349-4cff-a578-44fbd76baf3e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:24:16.145101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d73b6217-5349-4cff-a578-44fbd76baf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:16.248196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b6217-5349-4cff-a578-44fbd76baf3e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:24:16.317714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:16.319194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695fffd-a30a-4d82-8d74-8260ef2fbdc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:24:16.319798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695fffd-a30a-4d82-8d74-8260ef2fbdc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:16.320077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695fffd-a30a-4d82-8d74-8260ef2fbdc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:16.320627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695fffd-a30a-4d82-8d74-8260ef2fbdc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:16.323172 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1695fffd-a30a-4d82-8d74-8260ef2fbdc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:16.428174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695fffd-a30a-4d82-8d74-8260ef2fbdc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:24:16.429638 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695fffd-a30a-4d82-8d74-8260ef2fbdc3 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:24:16.448203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 732/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:24:18.662047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea5c49-c902-4edd-85fb-86fe5efd0d16 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:18.662587 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea5c49-c902-4edd-85fb-86fe5efd0d16 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:18.662587 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea5c49-c902-4edd-85fb-86fe5efd0d16 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:18.663834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 756/1488] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:24:18.669656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591f91-0274-48bb-bb61-b6ee58a5c27b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:18.670092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591f91-0274-48bb-bb61-b6ee58a5c27b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:18.670232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6591f91-0274-48bb-bb61-b6ee58a5c27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:18.734716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d6591f91-0274-48bb-bb61-b6ee58a5c27b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:24:18.735904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d6591f91-0274-48bb-bb61-b6ee58a5c27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:18.789411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:18.833231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 733/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:24:19.123955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea5c49-c902-4edd-85fb-86fe5efd0d16 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:24:19.125523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7aea5c49-c902-4edd-85fb-86fe5efd0d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:19.161312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:19.228868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea5c49-c902-4edd-85fb-86fe5efd0d16 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:24:19.302454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6197712-f182-4f3c-9331-09b08f345e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:24:19.302972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6197712-f182-4f3c-9331-09b08f345e3d service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:19.303264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6197712-f182-4f3c-9331-09b08f345e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:19.303529 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6197712-f182-4f3c-9331-09b08f345e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:19.305137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a6197712-f182-4f3c-9331-09b08f345e3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:19.384471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:19.415599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6197712-f182-4f3c-9331-09b08f345e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:24:19.417071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6197712-f182-4f3c-9331-09b08f345e3d service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:24:19.435633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 757/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:24:21.753984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db173d3-2386-427b-b82a-7cf966b04348 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:21.754558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db173d3-2386-427b-b82a-7cf966b04348 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:21.754558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db173d3-2386-427b-b82a-7cf966b04348 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:21.755876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 734/1491] 10.222.0.17 () {54 vars in 927 bytes} [Fri Sep 30 19:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:24:21.761618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb40ba-82b6-4a7c-a3bc-75657f1f0e97 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:21.762045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb40ba-82b6-4a7c-a3bc-75657f1f0e97 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:21.762159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb40ba-82b6-4a7c-a3bc-75657f1f0e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:21.825603 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-79fb40ba-82b6-4a7c-a3bc-75657f1f0e97 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:24:21.826801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-79fb40ba-82b6-4a7c-a3bc-75657f1f0e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:21.920590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 758/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Fri Sep 30 19:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:24:22.209181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db173d3-2386-427b-b82a-7cf966b04348 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:24:22.210765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2db173d3-2386-427b-b82a-7cf966b04348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:22.311983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db173d3-2386-427b-b82a-7cf966b04348 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:24:22.383399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68107675-6080-4334-9148-f8ae24086258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:24:22.384004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68107675-6080-4334-9148-f8ae24086258 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:24:22.384279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68107675-6080-4334-9148-f8ae24086258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:24:22.384581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68107675-6080-4334-9148-f8ae24086258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:24:22.386174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-68107675-6080-4334-9148-f8ae24086258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:24:22.395693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:22.497277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68107675-6080-4334-9148-f8ae24086258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:24:22.498623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68107675-6080-4334-9148-f8ae24086258 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:24:22.516702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 735/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:24:31.919085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=82 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:24:32.514395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=50 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:34.472195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823c831-c613-41b8-96c6-aa1a819aabb4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:34.472751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823c831-c613-41b8-96c6-aa1a819aabb4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:34.472751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823c831-c613-41b8-96c6-aa1a819aabb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:34.547109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b823c831-c613-41b8-96c6-aa1a819aabb4 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:25:34.548217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b823c831-c613-41b8-96c6-aa1a819aabb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:34.642591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 759/1494] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:25:34.651433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0def2-cb97-4a61-a54c-7b1dc4f6c5fe None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:34.651914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0def2-cb97-4a61-a54c-7b1dc4f6c5fe None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:34.652023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0def2-cb97-4a61-a54c-7b1dc4f6c5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:34.729652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-99d0def2-cb97-4a61-a54c-7b1dc4f6c5fe None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:25:34.730857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-99d0def2-cb97-4a61-a54c-7b1dc4f6c5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:34.831206 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 736/1495] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:25:34.836249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823c831-c613-41b8-96c6-aa1a819aabb4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:25:34.837479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b823c831-c613-41b8-96c6-aa1a819aabb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:34.848773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823c831-c613-41b8-96c6-aa1a819aabb4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:25:34.910255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:34.944334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b57e9-6f90-4e47-abe9-c3784cac7818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:25:34.944968 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b57e9-6f90-4e47-abe9-c3784cac7818 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:34.945238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b57e9-6f90-4e47-abe9-c3784cac7818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:34.945594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b57e9-6f90-4e47-abe9-c3784cac7818 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:34.946219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b57e9-6f90-4e47-abe9-c3784cac7818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:25:34.946511 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b57e9-6f90-4e47-abe9-c3784cac7818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:25:34.947502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-c51b57e9-6f90-4e47-abe9-c3784cac7818 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:25:34.947502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 30 19:25:34.947502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: Sep 30 19:25:34.948393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b57e9-6f90-4e47-abe9-c3784cac7818 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:25:34.959011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 760/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:25:34.967478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0def2-cb97-4a61-a54c-7b1dc4f6c5fe None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:25:34.976323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0def2-cb97-4a61-a54c-7b1dc4f6c5fe None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:25:35.036908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:35.071387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bd001-89c3-47c6-8609-e7fad893b0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:25:35.072045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bd001-89c3-47c6-8609-e7fad893b0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:35.072322 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bd001-89c3-47c6-8609-e7fad893b0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:35.072616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bd001-89c3-47c6-8609-e7fad893b0d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:35.073240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bd001-89c3-47c6-8609-e7fad893b0d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:25:35.075003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-005bd001-89c3-47c6-8609-e7fad893b0d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:25:35.075003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 30 19:25:35.075003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: Sep 30 19:25:35.076186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bd001-89c3-47c6-8609-e7fad893b0d9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:25:35.132544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 737/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:25:34 2022] POST /identity/v3/projects => generated 375 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:25:35.136787 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b57e9-6f90-4e47-abe9-c3784cac7818 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:25:35.138296 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c51b57e9-6f90-4e47-abe9-c3784cac7818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:35.171368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b57e9-6f90-4e47-abe9-c3784cac7818 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:25:35.264796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28ba39-cb62-40c4-9384-e5c4baa0c1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:25:35.265396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ba39-cb62-40c4-9384-e5c4baa0c1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:35.265670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ba39-cb62-40c4-9384-e5c4baa0c1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:35.265936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ba39-cb62-40c4-9384-e5c4baa0c1b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:35.266638 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28ba39-cb62-40c4-9384-e5c4baa0c1b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:25:35.267562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-fe28ba39-cb62-40c4-9384-e5c4baa0c1b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:25:35.267562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 30 19:25:35.267562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: Sep 30 19:25:35.268791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28ba39-cb62-40c4-9384-e5c4baa0c1b2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:25:35.321109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 761/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:25:35 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:25:35.325388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bd001-89c3-47c6-8609-e7fad893b0d9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:25:35.326927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-005bd001-89c3-47c6-8609-e7fad893b0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:35.392923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bd001-89c3-47c6-8609-e7fad893b0d9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:25:35.478277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:35.489144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cd416-1105-42b3-b424-ac64d52b7fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:25:35.489752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cd416-1105-42b3-b424-ac64d52b7fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:35.490025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cd416-1105-42b3-b424-ac64d52b7fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:35.490288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cd416-1105-42b3-b424-ac64d52b7fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:35.490947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cd416-1105-42b3-b424-ac64d52b7fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:25:35.491963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-9e1cd416-1105-42b3-b424-ac64d52b7fb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:25:35.491963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 30 19:25:35.491963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: Sep 30 19:25:35.492862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cd416-1105-42b3-b424-ac64d52b7fb9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:25:35.505594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 738/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:25:35.509555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28ba39-cb62-40c4-9384-e5c4baa0c1b2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:25:35.511098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fe28ba39-cb62-40c4-9384-e5c4baa0c1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:35.523056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28ba39-cb62-40c4-9384-e5c4baa0c1b2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:25:35.552713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:35.622959 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c933a-b335-4728-b3fa-eecc01eb6230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:25:35.623564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c933a-b335-4728-b3fa-eecc01eb6230 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:35.623842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c933a-b335-4728-b3fa-eecc01eb6230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:35.624114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c933a-b335-4728-b3fa-eecc01eb6230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:35.624777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c933a-b335-4728-b3fa-eecc01eb6230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:25:35.625782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-577c933a-b335-4728-b3fa-eecc01eb6230 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:25:35.625782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 30 19:25:35.625782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: Sep 30 19:25:35.626663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c933a-b335-4728-b3fa-eecc01eb6230 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:25:35.639519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 762/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:25:35.643728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cd416-1105-42b3-b424-ac64d52b7fb9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:25:35.645243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9e1cd416-1105-42b3-b424-ac64d52b7fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:35.678193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cd416-1105-42b3-b424-ac64d52b7fb9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:25:35.769880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45022a0-0b42-40a1-b40c-42e19e8ccba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:25:35.770544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45022a0-0b42-40a1-b40c-42e19e8ccba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:35.770820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45022a0-0b42-40a1-b40c-42e19e8ccba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:35.771088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45022a0-0b42-40a1-b40c-42e19e8ccba5 admin admin] PATH_INFO: `/v3/projects/f655676facf94981a596f6c0971f3d86/users/fc5e16f3bd4b480ab7bac52f82870431/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:35.793083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45022a0-0b42-40a1-b40c-42e19e8ccba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f655676facf94981a596f6c0971f3d86, user_id=fc5e16f3bd4b480ab7bac52f82870431, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:25:35.794109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-b45022a0-0b42-40a1-b40c-42e19e8ccba5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:25:35.794109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 30 19:25:35.794109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: Sep 30 19:25:35.796008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45022a0-0b42-40a1-b40c-42e19e8ccba5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:25:35.819949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 739/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:25:35 2022] PUT /identity/v3/projects/f655676facf94981a596f6c0971f3d86/users/fc5e16f3bd4b480ab7bac52f82870431/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:25:35.832286 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e96e8-5c80-4c71-9fcd-be15330ab43b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:35.832603 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e96e8-5c80-4c71-9fcd-be15330ab43b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:35.832877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e96e8-5c80-4c71-9fcd-be15330ab43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:35.888569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-091e96e8-5c80-4c71-9fcd-be15330ab43b None None] MFA Rules not processed for user `fc5e16f3bd4b480ab7bac52f82870431`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:25:35.889750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-091e96e8-5c80-4c71-9fcd-be15330ab43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:35.979040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 763/1502] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:25:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:25:35.993643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7b715-be44-4e3d-9661-8ccf5e75911b None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:35.993925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7b715-be44-4e3d-9661-8ccf5e75911b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:35.994194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7b715-be44-4e3d-9661-8ccf5e75911b None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:35.995573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 740/1503] 10.222.0.17 () {54 vars in 963 bytes} [Fri Sep 30 19:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:25:36.001047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140811c-258f-4f11-9a4f-818cd43e402a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:36.001326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140811c-258f-4f11-9a4f-818cd43e402a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:36.001623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140811c-258f-4f11-9a4f-818cd43e402a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:36.034812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:36.078876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8140811c-258f-4f11-9a4f-818cd43e402a None None] MFA Rules not processed for user `65d3020e1f6c414caabbaa185fb7ed48`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:25:36.080066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8140811c-258f-4f11-9a4f-818cd43e402a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:36.155354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 764/1504] 10.222.0.17 () {58 vars in 1060 bytes} [Fri Sep 30 19:25:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:25:36.164013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7b715-be44-4e3d-9661-8ccf5e75911b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:25:36.166060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d4a7b715-be44-4e3d-9661-8ccf5e75911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:36.216011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:36.269531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7b715-be44-4e3d-9661-8ccf5e75911b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:25:36.342004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162750e-5c05-4f49-a1db-2cf3da8aaebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:25:36.342606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162750e-5c05-4f49-a1db-2cf3da8aaebd service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:36.342879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162750e-5c05-4f49-a1db-2cf3da8aaebd service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:36.343145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162750e-5c05-4f49-a1db-2cf3da8aaebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:36.344762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8162750e-5c05-4f49-a1db-2cf3da8aaebd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:36.442133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162750e-5c05-4f49-a1db-2cf3da8aaebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:25:36.444199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162750e-5c05-4f49-a1db-2cf3da8aaebd service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:25:36.447623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:36.462302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 741/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:25:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:25:36.509762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425853e2-f907-41d1-ac00-b35ed86da759 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:36.510198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425853e2-f907-41d1-ac00-b35ed86da759 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:36.510308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425853e2-f907-41d1-ac00-b35ed86da759 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:36.511592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 765/1506] 10.222.0.17 () {52 vars in 830 bytes} [Fri Sep 30 19:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:25:36.515070 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162750e-5c05-4f49-a1db-2cf3da8aaebd service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:25:36.516276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8162750e-5c05-4f49-a1db-2cf3da8aaebd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:36.620936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162750e-5c05-4f49-a1db-2cf3da8aaebd service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:25:36.718746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:25:36.719443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:36.719614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:36.719878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] PATH_INFO: `/v3/users/fc5e16f3bd4b480ab7bac52f82870431` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:36.733935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc5e16f3bd4b480ab7bac52f82870431)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:25:36.734930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:25:36.734930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 30 19:25:36.734930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: Sep 30 19:25:36.736130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:25:36.746841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:36.816495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc5e16f3bd4b480ab7bac52f82870431'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:25:36.832231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc5e16f3bd4b480ab7bac52f82870431'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:25:36.832667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc5e16f3bd4b480ab7bac52f82870431'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:25:36.842486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc5e16f3bd4b480ab7bac52f82870431'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:25:36.879497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc5e16f3bd4b480ab7bac52f82870431'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:25:36.885264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 742/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:25:36 2022] DELETE /identity/v3/users/fc5e16f3bd4b480ab7bac52f82870431 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:25:36.889890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880ce76-7c34-4064-a458-924498fecab2 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:36.890168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880ce76-7c34-4064-a458-924498fecab2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:36.890466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880ce76-7c34-4064-a458-924498fecab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:36.948793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7880ce76-7c34-4064-a458-924498fecab2 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:25:36.949971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7880ce76-7c34-4064-a458-924498fecab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:37.025376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:37.039054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 766/1508] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:25:37.048695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:25:37.049908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:37.154088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707f288-d6b9-4784-92ac-dddd91cde396 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:25:37.211708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:37.235333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e690fce-2e31-458e-8e27-eff5d05a71e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:25:37.235948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e690fce-2e31-458e-8e27-eff5d05a71e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:37.236230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e690fce-2e31-458e-8e27-eff5d05a71e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:37.236531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e690fce-2e31-458e-8e27-eff5d05a71e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:37.238140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0e690fce-2e31-458e-8e27-eff5d05a71e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:37.359728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e690fce-2e31-458e-8e27-eff5d05a71e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:25:37.361109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e690fce-2e31-458e-8e27-eff5d05a71e1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:25:37.378772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 743/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:25:37.746584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880ce76-7c34-4064-a458-924498fecab2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:25:37.747900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7880ce76-7c34-4064-a458-924498fecab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:25:37.761720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880ce76-7c34-4064-a458-924498fecab2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:25:37.858875 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:25:37.859512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:25:37.859789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:25:37.860060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] PATH_INFO: `/v3/projects/f655676facf94981a596f6c0971f3d86` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:25:37.870572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f655676facf94981a596f6c0971f3d86)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:25:37.871600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:25:37.871600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 30 19:25:37.871600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: Sep 30 19:25:37.872826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:25:37.934306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f655676facf94981a596f6c0971f3d86', 'request_id': 'req-5538a88f-474b-46d3-bc26-0d308ee6faaa'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:25:37.940507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f655676facf94981a596f6c0971f3d86', 'request_id': 'req-5538a88f-474b-46d3-bc26-0d308ee6faaa'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:25:37.955655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f655676facf94981a596f6c0971f3d86', 'request_id': 'req-5538a88f-474b-46d3-bc26-0d308ee6faaa'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:25:37.983340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] The token cache is being invalidate because project f655676facf94981a596f6c0971f3d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:25:37.983648 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:25:37.984070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5538a88f-474b-46d3-bc26-0d308ee6faaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:25:37.985520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 767/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:25:37 2022] DELETE /identity/v3/projects/f655676facf94981a596f6c0971f3d86 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:25:47.376667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=62 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:25:47.981149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:27:04.515577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0416b-71b1-4af3-bd95-b787cba68b79 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:04.516115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0416b-71b1-4af3-bd95-b787cba68b79 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:04.516115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0416b-71b1-4af3-bd95-b787cba68b79 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:04.517440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:27:04.522950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67344e28-67f4-48b6-95fb-f94f8aa1c262 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:04.523416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67344e28-67f4-48b6-95fb-f94f8aa1c262 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:04.523578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67344e28-67f4-48b6-95fb-f94f8aa1c262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:04.600994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-67344e28-67f4-48b6-95fb-f94f8aa1c262 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:27:04.602187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-67344e28-67f4-48b6-95fb-f94f8aa1c262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:04.702783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:27:04.854574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0416b-71b1-4af3-bd95-b787cba68b79 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:27:04.856663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e1e0416b-71b1-4af3-bd95-b787cba68b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:04.974181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0416b-71b1-4af3-bd95-b787cba68b79 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:27:04.979717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf8c47-dc50-4ae5-833b-7c14bfb91a69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:27:04.980317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8c47-dc50-4ae5-833b-7c14bfb91a69 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:04.980641 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8c47-dc50-4ae5-833b-7c14bfb91a69 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:04.980910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8c47-dc50-4ae5-833b-7c14bfb91a69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:04.989492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf8c47-dc50-4ae5-833b-7c14bfb91a69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:27:04.991058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf8c47-dc50-4ae5-833b-7c14bfb91a69 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:27:05.000245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 745/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:27:06.065498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9699c-f7fa-4a64-9b28-cd7e03385dca None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:06.066056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9699c-f7fa-4a64-9b28-cd7e03385dca None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:06.066056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9699c-f7fa-4a64-9b28-cd7e03385dca None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:06.067917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:27:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:27:06.074081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05847ede-763d-4a00-bdf0-96dfe8cb4a68 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:06.074535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05847ede-763d-4a00-bdf0-96dfe8cb4a68 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:06.074668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05847ede-763d-4a00-bdf0-96dfe8cb4a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:06.094331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-05847ede-763d-4a00-bdf0-96dfe8cb4a68 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:27:06.095509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-05847ede-763d-4a00-bdf0-96dfe8cb4a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:06.103281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:27:06.251028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9699c-f7fa-4a64-9b28-cd7e03385dca None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:27:06.266591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9699c-f7fa-4a64-9b28-cd7e03385dca None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:27:06.272654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6a965-2eeb-421f-84b9-38e3c123301e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:27:06.273246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6a965-2eeb-421f-84b9-38e3c123301e service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:06.273519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6a965-2eeb-421f-84b9-38e3c123301e service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:06.273785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6a965-2eeb-421f-84b9-38e3c123301e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:06.281698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6a965-2eeb-421f-84b9-38e3c123301e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:27:06.283245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6a965-2eeb-421f-84b9-38e3c123301e service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:27:06.292548 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 770/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:27:06.332983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b9b19-f613-4732-ace5-4f993bf16a84 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:06.333285 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b9b19-f613-4732-ace5-4f993bf16a84 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:06.333561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b9b19-f613-4732-ace5-4f993bf16a84 None None] PATH_INFO: `/v3` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:06.335049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 747/1517] 10.222.0.36 () {54 vars in 925 bytes} [Fri Sep 30 19:27:06 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:27:06.341837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6fc87-cdb6-479e-9b8e-724a07874628 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:06.342118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6fc87-cdb6-479e-9b8e-724a07874628 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:06.342412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6fc87-cdb6-479e-9b8e-724a07874628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:06.409653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-69b6fc87-cdb6-479e-9b8e-724a07874628 None None] MFA Rules not processed for user `0a50fd417ae94859bb713b8b2066a9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:27:06.411144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-69b6fc87-cdb6-479e-9b8e-724a07874628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:06.429312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:27:06.483668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 771/1518] 10.222.0.36 () {58 vars in 1015 bytes} [Fri Sep 30 19:27:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:27:15.462351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f3de1-e660-44fd-8559-8f48e3542bc8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:15.462857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f3de1-e660-44fd-8559-8f48e3542bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:15.463054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f3de1-e660-44fd-8559-8f48e3542bc8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:15.464229 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 748/1519] 10.222.0.17 () {54 vars in 961 bytes} [Fri Sep 30 19:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:27:15.469263 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922c948-5c6b-4ab7-babe-01e1d2ec7888 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:15.469708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922c948-5c6b-4ab7-babe-01e1d2ec7888 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:15.469812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922c948-5c6b-4ab7-babe-01e1d2ec7888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:15.491315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7922c948-5c6b-4ab7-babe-01e1d2ec7888 None None] MFA Rules not processed for user `0a50fd417ae94859bb713b8b2066a9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:27:15.492668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7922c948-5c6b-4ab7-babe-01e1d2ec7888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:15.501153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 772/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Sep 30 19:27:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:27:15.508515 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f3de1-e660-44fd-8559-8f48e3542bc8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:27:15.521323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f3de1-e660-44fd-8559-8f48e3542bc8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:27:15.526593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9a1ad-55a4-45f4-8c83-aa716d78120a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:27:15.527167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9a1ad-55a4-45f4-8c83-aa716d78120a service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:15.527442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9a1ad-55a4-45f4-8c83-aa716d78120a service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:15.527708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9a1ad-55a4-45f4-8c83-aa716d78120a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:15.535819 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9a1ad-55a4-45f4-8c83-aa716d78120a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:27:15.538046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9a1ad-55a4-45f4-8c83-aa716d78120a service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:27:15.547320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 749/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:27:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:27:17.777969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454c34f-0a25-478b-b37f-8964d14851a0 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:17.778536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454c34f-0a25-478b-b37f-8964d14851a0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:17.778536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454c34f-0a25-478b-b37f-8964d14851a0 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:17.779849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:27:17.786321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b1a2f-7daf-4cc2-b78d-1f923b971e75 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:17.786763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b1a2f-7daf-4cc2-b78d-1f923b971e75 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:17.786873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b1a2f-7daf-4cc2-b78d-1f923b971e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:17.806104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-443b1a2f-7daf-4cc2-b78d-1f923b971e75 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:27:17.807263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-443b1a2f-7daf-4cc2-b78d-1f923b971e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:17.814842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:27:17.968833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454c34f-0a25-478b-b37f-8964d14851a0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:27:17.982736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454c34f-0a25-478b-b37f-8964d14851a0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:27:17.988038 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef7679-593d-4261-9521-50b8070c15b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:27:17.988639 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef7679-593d-4261-9521-50b8070c15b8 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:17.988912 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef7679-593d-4261-9521-50b8070c15b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:17.989177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef7679-593d-4261-9521-50b8070c15b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:17.996954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef7679-593d-4261-9521-50b8070c15b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:27:17.998507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef7679-593d-4261-9521-50b8070c15b8 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:27:18.007930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 774/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:27:19.063300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296e435-a237-45df-9a24-4f7bab41ee35 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:19.063884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296e435-a237-45df-9a24-4f7bab41ee35 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:19.063884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296e435-a237-45df-9a24-4f7bab41ee35 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:19.065189 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 19:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:27:19.071218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e559f1-97b7-463d-ad12-1735d3b6ea51 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:19.071637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e559f1-97b7-463d-ad12-1735d3b6ea51 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:19.071772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e559f1-97b7-463d-ad12-1735d3b6ea51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:19.091291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-82e559f1-97b7-463d-ad12-1735d3b6ea51 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:27:19.092507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-82e559f1-97b7-463d-ad12-1735d3b6ea51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:19.101021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 19:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:27:19.252779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296e435-a237-45df-9a24-4f7bab41ee35 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:27:19.267357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296e435-a237-45df-9a24-4f7bab41ee35 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:27:19.272724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d457b3b-02dd-4e63-b458-d6acc6db897c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:27:19.273297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d457b3b-02dd-4e63-b458-d6acc6db897c service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:19.273568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d457b3b-02dd-4e63-b458-d6acc6db897c service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:19.273833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d457b3b-02dd-4e63-b458-d6acc6db897c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:19.281551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d457b3b-02dd-4e63-b458-d6acc6db897c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:27:19.283101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d457b3b-02dd-4e63-b458-d6acc6db897c service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:27:19.292338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 752/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:27:23.394556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0a9fa-e8cd-4875-b0c5-79870bbc8156 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:23.395116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0a9fa-e8cd-4875-b0c5-79870bbc8156 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:23.395116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0a9fa-e8cd-4875-b0c5-79870bbc8156 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:23.396391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:27:23.402220 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06912b9b-0efd-4272-8736-d818bd84c8da None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:23.402642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06912b9b-0efd-4272-8736-d818bd84c8da None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:23.403256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06912b9b-0efd-4272-8736-d818bd84c8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:23.421971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-06912b9b-0efd-4272-8736-d818bd84c8da None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:27:23.423138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-06912b9b-0efd-4272-8736-d818bd84c8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:23.431319 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:27:23.719962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0a9fa-e8cd-4875-b0c5-79870bbc8156 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:27:23.721682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-75b0a9fa-e8cd-4875-b0c5-79870bbc8156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:23.799149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0a9fa-e8cd-4875-b0c5-79870bbc8156 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:27:23.802696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19303c26-64f4-4547-b5ce-307dbf4f893a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:27:23.803292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19303c26-64f4-4547-b5ce-307dbf4f893a service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:23.803569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19303c26-64f4-4547-b5ce-307dbf4f893a service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:23.803836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19303c26-64f4-4547-b5ce-307dbf4f893a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:23.812492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19303c26-64f4-4547-b5ce-307dbf4f893a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:27:23.813905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19303c26-64f4-4547-b5ce-307dbf4f893a service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:27:23.823472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 777/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:27:25.967695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286673ae-7426-4c67-90e9-f9dc47333fc0 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:25.968267 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286673ae-7426-4c67-90e9-f9dc47333fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:25.968267 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286673ae-7426-4c67-90e9-f9dc47333fc0 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:25.969603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:27:25.974846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb044-d071-4069-891d-2557c6f37417 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:25.975302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb044-d071-4069-891d-2557c6f37417 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:25.975386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cb044-d071-4069-891d-2557c6f37417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:25.995713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e62cb044-d071-4069-891d-2557c6f37417 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:27:25.996972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e62cb044-d071-4069-891d-2557c6f37417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:26.005276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:27:26.293513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286673ae-7426-4c67-90e9-f9dc47333fc0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:27:26.304045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286673ae-7426-4c67-90e9-f9dc47333fc0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:27:26.307478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9dd24-bde6-4321-8046-bd4760d88302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:27:26.308060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9dd24-bde6-4321-8046-bd4760d88302 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:26.308341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9dd24-bde6-4321-8046-bd4760d88302 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:26.308652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9dd24-bde6-4321-8046-bd4760d88302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:26.317260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9dd24-bde6-4321-8046-bd4760d88302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:27:26.318674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9dd24-bde6-4321-8046-bd4760d88302 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:27:26.328394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 755/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:27:29.990305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a80d0-9c7a-492b-ae7a-7515fefa237e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:29.990900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a80d0-9c7a-492b-ae7a-7515fefa237e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:29.990900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a80d0-9c7a-492b-ae7a-7515fefa237e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:29.992190 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:27:29.997936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ab841-f34e-4861-8fa7-83a3e15883be None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:29.998383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ab841-f34e-4861-8fa7-83a3e15883be None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:29.998478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ab841-f34e-4861-8fa7-83a3e15883be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:30.005856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=81 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:27:30.018484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c57ab841-f34e-4861-8fa7-83a3e15883be None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:27:30.019691 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c57ab841-f34e-4861-8fa7-83a3e15883be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:30.027374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:27:30.337898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a80d0-9c7a-492b-ae7a-7515fefa237e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:27:30.339559 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fe2a80d0-9c7a-492b-ae7a-7515fefa237e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:30.403014 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:27:30.416729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a80d0-9c7a-492b-ae7a-7515fefa237e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:27:30.420218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2730d-68e0-4bb7-90c9-08379399dc75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:27:30.420851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2730d-68e0-4bb7-90c9-08379399dc75 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:30.421130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2730d-68e0-4bb7-90c9-08379399dc75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:30.421399 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2730d-68e0-4bb7-90c9-08379399dc75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:30.430073 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2730d-68e0-4bb7-90c9-08379399dc75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:27:30.431984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2730d-68e0-4bb7-90c9-08379399dc75 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:27:30.441125 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 780/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:27:32.565577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0726f3b-d570-4b01-b508-3d22ecf05afa None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:32.566137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0726f3b-d570-4b01-b508-3d22ecf05afa None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:32.566137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0726f3b-d570-4b01-b508-3d22ecf05afa None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:32.567446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 19:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:27:32.572703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223647-8fa1-4c7a-84ef-d45e73cdebaf None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:32.573154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223647-8fa1-4c7a-84ef-d45e73cdebaf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:32.573265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223647-8fa1-4c7a-84ef-d45e73cdebaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:32.593027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-30223647-8fa1-4c7a-84ef-d45e73cdebaf None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:27:32.594196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-30223647-8fa1-4c7a-84ef-d45e73cdebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:32.601755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 19:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:27:32.900756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0726f3b-d570-4b01-b508-3d22ecf05afa None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:27:32.902390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b0726f3b-d570-4b01-b508-3d22ecf05afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:27:32.915206 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0726f3b-d570-4b01-b508-3d22ecf05afa None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:27:32.918631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d6f03-78ac-4b1e-b334-5cae123faa37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:27:32.919212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d6f03-78ac-4b1e-b334-5cae123faa37 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:27:32.919489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d6f03-78ac-4b1e-b334-5cae123faa37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:27:32.919759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d6f03-78ac-4b1e-b334-5cae123faa37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:27:32.928335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d6f03-78ac-4b1e-b334-5cae123faa37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:27:32.929771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d6f03-78ac-4b1e-b334-5cae123faa37 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:27:32.938877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 758/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:27:42.588128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=11 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:27:42.934850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=8 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:57.012454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b222b-b7dc-43ee-a425-5be903f76ab0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.013127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b222b-b7dc-43ee-a425-5be903f76ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.013127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b222b-b7dc-43ee-a425-5be903f76ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.036420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9c1b222b-b7dc-43ee-a425-5be903f76ab0 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.038599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9c1b222b-b7dc-43ee-a425-5be903f76ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.051296 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 782/1540] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.060875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8d547-993d-4465-af50-edebcd821286 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.061495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8d547-993d-4465-af50-edebcd821286 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.061495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8d547-993d-4465-af50-edebcd821286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.089139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-99a8d547-993d-4465-af50-edebcd821286 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.090411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-99a8d547-993d-4465-af50-edebcd821286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.104607 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 759/1541] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.109357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b222b-b7dc-43ee-a425-5be903f76ab0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.125428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b222b-b7dc-43ee-a425-5be903f76ab0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:57.138428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7c55d-4338-4635-b81f-807bed337642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:57.139065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7c55d-4338-4635-b81f-807bed337642 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.139336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7c55d-4338-4635-b81f-807bed337642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.139597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7c55d-4338-4635-b81f-807bed337642 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.140260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7c55d-4338-4635-b81f-807bed337642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:28:57.140577 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7c55d-4338-4635-b81f-807bed337642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:57.142222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7c55d-4338-4635-b81f-807bed337642 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:57.161491 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 783/1542] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:28:57.170382 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8d547-993d-4465-af50-edebcd821286 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.193718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8d547-993d-4465-af50-edebcd821286 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:57.203234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca406d3-4cd2-4cb7-8520-825db9e6050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:57.208395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca406d3-4cd2-4cb7-8520-825db9e6050a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.208395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca406d3-4cd2-4cb7-8520-825db9e6050a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.208395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca406d3-4cd2-4cb7-8520-825db9e6050a admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.208395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca406d3-4cd2-4cb7-8520-825db9e6050a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:57.208395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca406d3-4cd2-4cb7-8520-825db9e6050a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:57.262796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 760/1543] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:57.266830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7c55d-4338-4635-b81f-807bed337642 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.284034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7c55d-4338-4635-b81f-807bed337642 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:57.324214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aacca-5393-4061-bdb9-5cab054a18f3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.324604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aacca-5393-4061-bdb9-5cab054a18f3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.325398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958aacca-5393-4061-bdb9-5cab054a18f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.364734 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-958aacca-5393-4061-bdb9-5cab054a18f3 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.366504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-958aacca-5393-4061-bdb9-5cab054a18f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.387381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211e8ce-dbed-4261-b307-98a2df6afe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:57.388210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211e8ce-dbed-4261-b307-98a2df6afe35 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.388534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211e8ce-dbed-4261-b307-98a2df6afe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.388805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211e8ce-dbed-4261-b307-98a2df6afe35 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.389450 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2211e8ce-dbed-4261-b307-98a2df6afe35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:57.391370 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2211e8ce-dbed-4261-b307-98a2df6afe35 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:57.455913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 784/1544] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:28:57.457236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3c8b-484f-4bf0-9cfc-02fb7d4988fd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.457843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3c8b-484f-4bf0-9cfc-02fb7d4988fd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.458274 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3c8b-484f-4bf0-9cfc-02fb7d4988fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.492886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-442c3c8b-484f-4bf0-9cfc-02fb7d4988fd None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.494366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-442c3c8b-484f-4bf0-9cfc-02fb7d4988fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.507557 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 785/1545] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.508664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c3c8b-484f-4bf0-9cfc-02fb7d4988fd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.518475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 761/1546] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:28:57.519688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82826-daeb-4e7e-9199-b1e4e0aaefca None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.519983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82826-daeb-4e7e-9199-b1e4e0aaefca None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.520576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82826-daeb-4e7e-9199-b1e4e0aaefca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.531227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c3c8b-484f-4bf0-9cfc-02fb7d4988fd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:57.540434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b1c5d-0be4-4243-b6d9-e6af00227df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:57.541409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b1c5d-0be4-4243-b6d9-e6af00227df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.541857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b1c5d-0be4-4243-b6d9-e6af00227df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.542282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b1c5d-0be4-4243-b6d9-e6af00227df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.543094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b1c5d-0be4-4243-b6d9-e6af00227df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:57.545005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b1c5d-0be4-4243-b6d9-e6af00227df1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:57.566874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cac82826-daeb-4e7e-9199-b1e4e0aaefca None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.568178 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cac82826-daeb-4e7e-9199-b1e4e0aaefca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.569545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 786/1547] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:57.570850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9435737-fdf3-4ec3-a1e8-c6e2ca78b2ab None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.571132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9435737-fdf3-4ec3-a1e8-c6e2ca78b2ab None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.571404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9435737-fdf3-4ec3-a1e8-c6e2ca78b2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.586721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 762/1548] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.588318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bc296-1032-4303-8d67-7fff3af8561a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.588837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bc296-1032-4303-8d67-7fff3af8561a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.589385 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bc296-1032-4303-8d67-7fff3af8561a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.598153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b9435737-fdf3-4ec3-a1e8-c6e2ca78b2ab None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.600979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b9435737-fdf3-4ec3-a1e8-c6e2ca78b2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.610189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 787/1549] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.611846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795134-87af-453d-bd36-99575add7037 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.612322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795134-87af-453d-bd36-99575add7037 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.612842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795134-87af-453d-bd36-99575add7037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.613129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3a5bc296-1032-4303-8d67-7fff3af8561a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.614403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3a5bc296-1032-4303-8d67-7fff3af8561a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.625633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 763/1550] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.626501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bc296-1032-4303-8d67-7fff3af8561a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.634934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8c795134-87af-453d-bd36-99575add7037 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.636184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8c795134-87af-453d-bd36-99575add7037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.637157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bc296-1032-4303-8d67-7fff3af8561a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:57.641684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5804df-7379-4622-b287-04f298b7f515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:57.642298 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5804df-7379-4622-b287-04f298b7f515 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.642574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5804df-7379-4622-b287-04f298b7f515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.642897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5804df-7379-4622-b287-04f298b7f515 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.643585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5804df-7379-4622-b287-04f298b7f515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:57.644330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 788/1551] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.648528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5804df-7379-4622-b287-04f298b7f515 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:57.648640 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5ed4c-c2f6-424f-b758-4c4528266c11 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.648640 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5ed4c-c2f6-424f-b758-4c4528266c11 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.648640 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5ed4c-c2f6-424f-b758-4c4528266c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.659715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 764/1552] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:57.659715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92004-00ae-46eb-a31e-3d033779e682 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.659715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92004-00ae-46eb-a31e-3d033779e682 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.659715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92004-00ae-46eb-a31e-3d033779e682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.666670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-cca5ed4c-c2f6-424f-b758-4c4528266c11 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.667876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cca5ed4c-c2f6-424f-b758-4c4528266c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.675796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 789/1553] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.678316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044e3ff-1b89-4e51-91d3-fe1b8d2e3624 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.678607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044e3ff-1b89-4e51-91d3-fe1b8d2e3624 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.678983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044e3ff-1b89-4e51-91d3-fe1b8d2e3624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.679743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0ee92004-00ae-46eb-a31e-3d033779e682 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.681078 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0ee92004-00ae-46eb-a31e-3d033779e682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.689658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 765/1554] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.690440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee92004-00ae-46eb-a31e-3d033779e682 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.700265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c044e3ff-1b89-4e51-91d3-fe1b8d2e3624 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.700567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee92004-00ae-46eb-a31e-3d033779e682 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:57.701536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c044e3ff-1b89-4e51-91d3-fe1b8d2e3624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.705490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e2c91-7c27-479b-a4ce-2873d28fa654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:57.706126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2c91-7c27-479b-a4ce-2873d28fa654 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.706427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2c91-7c27-479b-a4ce-2873d28fa654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.706722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2c91-7c27-479b-a4ce-2873d28fa654 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.707491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e2c91-7c27-479b-a4ce-2873d28fa654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:28:57.707841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e2c91-7c27-479b-a4ce-2873d28fa654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:57.709038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-631e2c91-7c27-479b-a4ce-2873d28fa654 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:28:57.709038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 30 19:28:57.709038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: Sep 30 19:28:57.709603 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 790/1555] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.709948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e2c91-7c27-479b-a4ce-2873d28fa654 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:57.710550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f9dea-773b-4098-a22b-00520918e596 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.710825 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f9dea-773b-4098-a22b-00520918e596 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.711088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f9dea-773b-4098-a22b-00520918e596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.728106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 766/1556] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:28:57.729478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e2c91-7c27-479b-a4ce-2873d28fa654 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.730739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2d2f9dea-773b-4098-a22b-00520918e596 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.732105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2d2f9dea-773b-4098-a22b-00520918e596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.741313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 791/1557] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.745072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e2c91-7c27-479b-a4ce-2873d28fa654 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:57.745726 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4976681-29d0-499e-b247-9694575220e7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.745947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4976681-29d0-499e-b247-9694575220e7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.746325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4976681-29d0-499e-b247-9694575220e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.750457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256573e-c3e7-4e82-a974-5740b0fc106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:57.751055 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256573e-c3e7-4e82-a974-5740b0fc106d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.751334 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256573e-c3e7-4e82-a974-5740b0fc106d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.751605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256573e-c3e7-4e82-a974-5740b0fc106d admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.752283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256573e-c3e7-4e82-a974-5740b0fc106d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:28:57.752619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256573e-c3e7-4e82-a974-5740b0fc106d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:57.754222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256573e-c3e7-4e82-a974-5740b0fc106d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:57.766756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d4976681-29d0-499e-b247-9694575220e7 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.766932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 767/1558] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:28:57.767720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256573e-c3e7-4e82-a974-5740b0fc106d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.768161 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d4976681-29d0-499e-b247-9694575220e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.777558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 792/1559] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.778905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff881cb-7ea2-4ca2-87af-b46d8fe2edf0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.779209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff881cb-7ea2-4ca2-87af-b46d8fe2edf0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.779590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff881cb-7ea2-4ca2-87af-b46d8fe2edf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.779855 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256573e-c3e7-4e82-a974-5740b0fc106d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:57.785654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ca888-9513-4eb7-8761-7cdb0867943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:57.786309 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca888-9513-4eb7-8761-7cdb0867943c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.786588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca888-9513-4eb7-8761-7cdb0867943c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.787390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca888-9513-4eb7-8761-7cdb0867943c admin admin] PATH_INFO: `/v3/projects/d9758637e30e4089b394ed6ea5c2cf3f/users/f1bb718f9153493ca6dda343d461c5ab/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.801097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-dff881cb-7ea2-4ca2-87af-b46d8fe2edf0 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.802175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ca888-9513-4eb7-8761-7cdb0867943c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9758637e30e4089b394ed6ea5c2cf3f, user_id=f1bb718f9153493ca6dda343d461c5ab, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:57.802339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dff881cb-7ea2-4ca2-87af-b46d8fe2edf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.805000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ca888-9513-4eb7-8761-7cdb0867943c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:57.810912 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 793/1560] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.811756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff881cb-7ea2-4ca2-87af-b46d8fe2edf0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.819074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 768/1561] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:28:57 2022] PUT /identity/v3/projects/d9758637e30e4089b394ed6ea5c2cf3f/users/f1bb718f9153493ca6dda343d461c5ab/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:28:57.820644 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f46f-1653-4f1c-a4a4-7cd4f6771020 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.820931 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f46f-1653-4f1c-a4a4-7cd4f6771020 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.821205 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f46f-1653-4f1c-a4a4-7cd4f6771020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.821812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff881cb-7ea2-4ca2-87af-b46d8fe2edf0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:57.840692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-96c0f46f-1653-4f1c-a4a4-7cd4f6771020 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.841956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-96c0f46f-1653-4f1c-a4a4-7cd4f6771020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.880284 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:57.881792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4498dab-1dd3-454d-a88b-e98b042f47fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:57.882594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4498dab-1dd3-454d-a88b-e98b042f47fe admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.883104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4498dab-1dd3-454d-a88b-e98b042f47fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.883224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4498dab-1dd3-454d-a88b-e98b042f47fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.883840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4498dab-1dd3-454d-a88b-e98b042f47fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:28:57.884176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4498dab-1dd3-454d-a88b-e98b042f47fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:57.886071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4498dab-1dd3-454d-a88b-e98b042f47fe admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:57.895111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:57.906181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 794/1562] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:28:57.906181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4498dab-1dd3-454d-a88b-e98b042f47fe admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.910362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4498dab-1dd3-454d-a88b-e98b042f47fe admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:57.911356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 769/1563] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.913706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b143f-6fa2-40f8-a825-efc5beed766e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.914015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b143f-6fa2-40f8-a825-efc5beed766e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.914246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b143f-6fa2-40f8-a825-efc5beed766e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.915769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a2476-71a8-42fd-842c-4444b8c4c396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:57.916447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a2476-71a8-42fd-842c-4444b8c4c396 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:57.916731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a2476-71a8-42fd-842c-4444b8c4c396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:57.917033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a2476-71a8-42fd-842c-4444b8c4c396 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:57.917650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a2476-71a8-42fd-842c-4444b8c4c396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:57.918719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-c40a2476-71a8-42fd-842c-4444b8c4c396 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:28:57.918719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 30 19:28:57.918719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: Sep 30 19:28:57.919950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a2476-71a8-42fd-842c-4444b8c4c396 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:57.933871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cc4b143f-6fa2-40f8-a825-efc5beed766e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:57.935046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cc4b143f-6fa2-40f8-a825-efc5beed766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:57.944105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 770/1564] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:57.947824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b143f-6fa2-40f8-a825-efc5beed766e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.955629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 795/1565] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:57.955629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a2476-71a8-42fd-842c-4444b8c4c396 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:57.955880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b143f-6fa2-40f8-a825-efc5beed766e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.011743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558dece-1e52-4ea5-a333-c5c59f2eae0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.012488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558dece-1e52-4ea5-a333-c5c59f2eae0e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.012770 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558dece-1e52-4ea5-a333-c5c59f2eae0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.013041 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558dece-1e52-4ea5-a333-c5c59f2eae0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.013661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558dece-1e52-4ea5-a333-c5c59f2eae0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.015566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558dece-1e52-4ea5-a333-c5c59f2eae0e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.018532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a2476-71a8-42fd-842c-4444b8c4c396 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.023731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88387bf1-d237-461b-ada1-c9aca903eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.024323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88387bf1-d237-461b-ada1-c9aca903eb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.024625 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88387bf1-d237-461b-ada1-c9aca903eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.024894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88387bf1-d237-461b-ada1-c9aca903eb0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.025566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88387bf1-d237-461b-ada1-c9aca903eb0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:28:58.025890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88387bf1-d237-461b-ada1-c9aca903eb0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.027527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88387bf1-d237-461b-ada1-c9aca903eb0a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.047128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 796/1566] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:28:58.047128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88387bf1-d237-461b-ada1-c9aca903eb0a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.047409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 771/1567] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:28:57 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.047409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445840b-8a80-4ef7-86cc-66b20d7495b5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.047409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445840b-8a80-4ef7-86cc-66b20d7495b5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.047409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445840b-8a80-4ef7-86cc-66b20d7495b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.050506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88387bf1-d237-461b-ada1-c9aca903eb0a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.065404 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3445840b-8a80-4ef7-86cc-66b20d7495b5 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:58.066639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3445840b-8a80-4ef7-86cc-66b20d7495b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:58.106238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e7709-63e5-4818-ab52-cdef0051443f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.106924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7709-63e5-4818-ab52-cdef0051443f admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.107203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7709-63e5-4818-ab52-cdef0051443f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.107485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7709-63e5-4818-ab52-cdef0051443f admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.108163 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e7709-63e5-4818-ab52-cdef0051443f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:28:58.108510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e7709-63e5-4818-ab52-cdef0051443f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.110320 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e7709-63e5-4818-ab52-cdef0051443f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.123387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 797/1568] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:28:58.124333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e7709-63e5-4818-ab52-cdef0051443f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.125888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 772/1569] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:58.126903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445840b-8a80-4ef7-86cc-66b20d7495b5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.136297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e7709-63e5-4818-ab52-cdef0051443f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.138632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445840b-8a80-4ef7-86cc-66b20d7495b5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.141340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81527443-5504-4c19-99cb-3ec90f49e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.141947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81527443-5504-4c19-99cb-3ec90f49e1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.142225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81527443-5504-4c19-99cb-3ec90f49e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.142634 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81527443-5504-4c19-99cb-3ec90f49e1d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.143319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81527443-5504-4c19-99cb-3ec90f49e1d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.143432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f0a65-8312-43f3-8fa0-cd4b97f1a6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.143913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f0a65-8312-43f3-8fa0-cd4b97f1a6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.144190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f0a65-8312-43f3-8fa0-cd4b97f1a6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.144485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f0a65-8312-43f3-8fa0-cd4b97f1a6d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.145124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f0a65-8312-43f3-8fa0-cd4b97f1a6d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:28:58.145362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81527443-5504-4c19-99cb-3ec90f49e1d5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.145471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f0a65-8312-43f3-8fa0-cd4b97f1a6d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.147487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f0a65-8312-43f3-8fa0-cd4b97f1a6d0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.167291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 773/1570] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:28:58.167291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f0a65-8312-43f3-8fa0-cd4b97f1a6d0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.178375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f0a65-8312-43f3-8fa0-cd4b97f1a6d0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.182416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 798/1571] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.182416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81527443-5504-4c19-99cb-3ec90f49e1d5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.227745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:58.228031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:58.241258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80028d-5f7d-4124-8b5f-0278dd68f25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.241867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80028d-5f7d-4124-8b5f-0278dd68f25e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.242143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80028d-5f7d-4124-8b5f-0278dd68f25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.242418 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80028d-5f7d-4124-8b5f-0278dd68f25e admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.243086 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80028d-5f7d-4124-8b5f-0278dd68f25e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:28:58.243387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80028d-5f7d-4124-8b5f-0278dd68f25e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.245018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80028d-5f7d-4124-8b5f-0278dd68f25e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.246316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81527443-5504-4c19-99cb-3ec90f49e1d5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.251248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd6ebe-2644-4602-9f3c-3d0a82a9cba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.251894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6ebe-2644-4602-9f3c-3d0a82a9cba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.252168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6ebe-2644-4602-9f3c-3d0a82a9cba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.252475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6ebe-2644-4602-9f3c-3d0a82a9cba9 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.253098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd6ebe-2644-4602-9f3c-3d0a82a9cba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.254982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd6ebe-2644-4602-9f3c-3d0a82a9cba9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.256565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 774/1572] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:28:58.257337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80028d-5f7d-4124-8b5f-0278dd68f25e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.266800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80028d-5f7d-4124-8b5f-0278dd68f25e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.274712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b066efb-c25d-478a-93ce-fb09a57c9837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.275647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b066efb-c25d-478a-93ce-fb09a57c9837 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.275932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b066efb-c25d-478a-93ce-fb09a57c9837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.276296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b066efb-c25d-478a-93ce-fb09a57c9837 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.277181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b066efb-c25d-478a-93ce-fb09a57c9837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.279332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b066efb-c25d-478a-93ce-fb09a57c9837 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.283461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 799/1573] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.284570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd6ebe-2644-4602-9f3c-3d0a82a9cba9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.294772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd6ebe-2644-4602-9f3c-3d0a82a9cba9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.300393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e869a-9bee-4041-9078-9c03f77b2074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.300953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e869a-9bee-4041-9078-9c03f77b2074 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.301233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e869a-9bee-4041-9078-9c03f77b2074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.301528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e869a-9bee-4041-9078-9c03f77b2074 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.302198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e869a-9bee-4041-9078-9c03f77b2074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.304324 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e869a-9bee-4041-9078-9c03f77b2074 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.308766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 775/1574] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.309785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b066efb-c25d-478a-93ce-fb09a57c9837 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.327812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 800/1575] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.328674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e869a-9bee-4041-9078-9c03f77b2074 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.376776 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b066efb-c25d-478a-93ce-fb09a57c9837 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.382020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fb6fc-f1f6-4ea4-814d-89df414f27de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.382642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fb6fc-f1f6-4ea4-814d-89df414f27de admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.382913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fb6fc-f1f6-4ea4-814d-89df414f27de admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.383239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fb6fc-f1f6-4ea4-814d-89df414f27de admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.383357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e869a-9bee-4041-9078-9c03f77b2074 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.383870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fb6fc-f1f6-4ea4-814d-89df414f27de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.385794 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fb6fc-f1f6-4ea4-814d-89df414f27de admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.417307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 776/1576] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.418172 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fb6fc-f1f6-4ea4-814d-89df414f27de admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.442541 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa2b91-f2c5-460c-af9c-37da9b788f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.443636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa2b91-f2c5-460c-af9c-37da9b788f45 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.443907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa2b91-f2c5-460c-af9c-37da9b788f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.444168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa2b91-f2c5-460c-af9c-37da9b788f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.453040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa2b91-f2c5-460c-af9c-37da9b788f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.454452 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa2b91-f2c5-460c-af9c-37da9b788f45 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.464161 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 801/1577] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:28:58.464964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa2b91-f2c5-460c-af9c-37da9b788f45 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.474355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa2b91-f2c5-460c-af9c-37da9b788f45 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.486497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fb6fc-f1f6-4ea4-814d-89df414f27de admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.493690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f665b7-b6de-448b-8ca2-caccafe7a7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.494317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f665b7-b6de-448b-8ca2-caccafe7a7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.494599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f665b7-b6de-448b-8ca2-caccafe7a7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.494870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f665b7-b6de-448b-8ca2-caccafe7a7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.495493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f665b7-b6de-448b-8ca2-caccafe7a7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.497432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f665b7-b6de-448b-8ca2-caccafe7a7a3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.528894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 777/1578] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/projects => generated 389 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.530063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f665b7-b6de-448b-8ca2-caccafe7a7a3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.542439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca16299-787d-4046-b1f9-056fb3775bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.542732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16299-787d-4046-b1f9-056fb3775bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.542971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16299-787d-4046-b1f9-056fb3775bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.543240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16299-787d-4046-b1f9-056fb3775bdf admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.543883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca16299-787d-4046-b1f9-056fb3775bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.545964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca16299-787d-4046-b1f9-056fb3775bdf admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.572195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 802/1579] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.572195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca16299-787d-4046-b1f9-056fb3775bdf admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.578615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:58.579114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:58.583034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca16299-787d-4046-b1f9-056fb3775bdf admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.611742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f665b7-b6de-448b-8ca2-caccafe7a7a3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.618158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935d4ed-2cfc-402f-add9-caa36f977803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.619161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d4ed-2cfc-402f-add9-caa36f977803 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.619360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d4ed-2cfc-402f-add9-caa36f977803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.620215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d4ed-2cfc-402f-add9-caa36f977803 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.621279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935d4ed-2cfc-402f-add9-caa36f977803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.623324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935d4ed-2cfc-402f-add9-caa36f977803 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.650034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadef6e5-f413-4f24-ae56-25cd1d732336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.650432 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadef6e5-f413-4f24-ae56-25cd1d732336 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.650730 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadef6e5-f413-4f24-ae56-25cd1d732336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.651021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadef6e5-f413-4f24-ae56-25cd1d732336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.651679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadef6e5-f413-4f24-ae56-25cd1d732336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.653378 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadef6e5-f413-4f24-ae56-25cd1d732336 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.656837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 778/1580] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.662956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935d4ed-2cfc-402f-add9-caa36f977803 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.668417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 803/1581] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:58.668417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadef6e5-f413-4f24-ae56-25cd1d732336 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.682048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadef6e5-f413-4f24-ae56-25cd1d732336 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.746816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935d4ed-2cfc-402f-add9-caa36f977803 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.749966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446ce44-c469-4b00-ae55-7658f241f99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.750760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446ce44-c469-4b00-ae55-7658f241f99a admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.751023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446ce44-c469-4b00-ae55-7658f241f99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.751200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446ce44-c469-4b00-ae55-7658f241f99a admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.752048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446ce44-c469-4b00-ae55-7658f241f99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.753808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6d264-40e4-404a-93e8-f2d42febf87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.754024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446ce44-c469-4b00-ae55-7658f241f99a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.754621 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6d264-40e4-404a-93e8-f2d42febf87e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.754966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6d264-40e4-404a-93e8-f2d42febf87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.755317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6d264-40e4-404a-93e8-f2d42febf87e admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.756048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6d264-40e4-404a-93e8-f2d42febf87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.757379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-30d6d264-40e4-404a-93e8-f2d42febf87e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:28:58.757379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 30 19:28:58.757379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: Sep 30 19:28:58.758743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6d264-40e4-404a-93e8-f2d42febf87e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.769883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 804/1582] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:58.770749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446ce44-c469-4b00-ae55-7658f241f99a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.786054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446ce44-c469-4b00-ae55-7658f241f99a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.786217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 779/1583] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.786217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6d264-40e4-404a-93e8-f2d42febf87e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.792546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368cea8-44f9-49a2-89f2-e214dcd5f18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.794312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368cea8-44f9-49a2-89f2-e214dcd5f18b admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.794843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368cea8-44f9-49a2-89f2-e214dcd5f18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.795313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368cea8-44f9-49a2-89f2-e214dcd5f18b admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.796447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368cea8-44f9-49a2-89f2-e214dcd5f18b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.798270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6d264-40e4-404a-93e8-f2d42febf87e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.800630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368cea8-44f9-49a2-89f2-e214dcd5f18b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.805133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b8dbc-bf58-41a6-9469-35f376ab68ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.805874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b8dbc-bf58-41a6-9469-35f376ab68ef admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.806261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b8dbc-bf58-41a6-9469-35f376ab68ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.806752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b8dbc-bf58-41a6-9469-35f376ab68ef admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.807469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b8dbc-bf58-41a6-9469-35f376ab68ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.810789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b8dbc-bf58-41a6-9469-35f376ab68ef admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.829356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 805/1584] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/users => generated 297 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.830451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368cea8-44f9-49a2-89f2-e214dcd5f18b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.842853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368cea8-44f9-49a2-89f2-e214dcd5f18b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.844575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 780/1585] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/users => generated 295 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.848753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b8dbc-bf58-41a6-9469-35f376ab68ef admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.851616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d0ada-39ab-487a-a505-63cee9a247e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.852599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d0ada-39ab-487a-a505-63cee9a247e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.853055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d0ada-39ab-487a-a505-63cee9a247e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.853495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d0ada-39ab-487a-a505-63cee9a247e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.854490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d0ada-39ab-487a-a505-63cee9a247e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.856512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d0ada-39ab-487a-a505-63cee9a247e0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.861183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b8dbc-bf58-41a6-9469-35f376ab68ef admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.867420 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7807925-fac7-4224-80c8-903793372067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.868306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7807925-fac7-4224-80c8-903793372067 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.868683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7807925-fac7-4224-80c8-903793372067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.869028 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7807925-fac7-4224-80c8-903793372067 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.869782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7807925-fac7-4224-80c8-903793372067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.870738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 806/1586] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:58.871869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d0ada-39ab-487a-a505-63cee9a247e0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.872187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7807925-fac7-4224-80c8-903793372067 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.883811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d0ada-39ab-487a-a505-63cee9a247e0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.892349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e2178-df36-4ed0-9e5b-c259523a1bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.893129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e2178-df36-4ed0-9e5b-c259523a1bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.893404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e2178-df36-4ed0-9e5b-c259523a1bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.893672 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e2178-df36-4ed0-9e5b-c259523a1bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.894423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e2178-df36-4ed0-9e5b-c259523a1bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.896913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e2178-df36-4ed0-9e5b-c259523a1bc4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.906606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 781/1587] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:28:58 2022] POST /identity/v3/users => generated 293 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:28:58.907316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7807925-fac7-4224-80c8-903793372067 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.910895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 807/1588] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:58.912472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e2178-df36-4ed0-9e5b-c259523a1bc4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.920567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7807925-fac7-4224-80c8-903793372067 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.924944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e2178-df36-4ed0-9e5b-c259523a1bc4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.925939 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c16ec-3867-4f16-934a-eb1996df39f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.926624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c16ec-3867-4f16-934a-eb1996df39f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.926974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c16ec-3867-4f16-934a-eb1996df39f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.927876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c16ec-3867-4f16-934a-eb1996df39f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.928648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c16ec-3867-4f16-934a-eb1996df39f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.930355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c16ec-3867-4f16-934a-eb1996df39f5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.932469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f923f-87a6-4a67-8855-40ba92f5bfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.933370 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f923f-87a6-4a67-8855-40ba92f5bfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.933646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f923f-87a6-4a67-8855-40ba92f5bfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.934008 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f923f-87a6-4a67-8855-40ba92f5bfa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.934729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f923f-87a6-4a67-8855-40ba92f5bfa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.936660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f923f-87a6-4a67-8855-40ba92f5bfa0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.942353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 782/1589] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:58.943576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c16ec-3867-4f16-934a-eb1996df39f5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.953343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 808/1590] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:58.954498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f923f-87a6-4a67-8855-40ba92f5bfa0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.957658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c16ec-3867-4f16-934a-eb1996df39f5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.963223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc446c6-8111-4038-b0e6-3e7ebae5385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.963984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc446c6-8111-4038-b0e6-3e7ebae5385b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.964354 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc446c6-8111-4038-b0e6-3e7ebae5385b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.964745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc446c6-8111-4038-b0e6-3e7ebae5385b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.965490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc446c6-8111-4038-b0e6-3e7ebae5385b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.966476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f923f-87a6-4a67-8855-40ba92f5bfa0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.967325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc446c6-8111-4038-b0e6-3e7ebae5385b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.971930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecf90a-f0c7-4d1c-a1ac-6ddbad45588b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:58.972817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecf90a-f0c7-4d1c-a1ac-6ddbad45588b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:58.973570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecf90a-f0c7-4d1c-a1ac-6ddbad45588b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:58.973944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecf90a-f0c7-4d1c-a1ac-6ddbad45588b admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:58.974670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecf90a-f0c7-4d1c-a1ac-6ddbad45588b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:58.976823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecf90a-f0c7-4d1c-a1ac-6ddbad45588b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:58.978331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 783/1591] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:58.979381 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc446c6-8111-4038-b0e6-3e7ebae5385b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:58.994156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc446c6-8111-4038-b0e6-3e7ebae5385b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:58.995367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 809/1592] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:58.996350 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecf90a-f0c7-4d1c-a1ac-6ddbad45588b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.002017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929686c-6538-4cf4-84a9-3a228a676f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.002699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929686c-6538-4cf4-84a9-3a228a676f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.003424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929686c-6538-4cf4-84a9-3a228a676f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.003424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929686c-6538-4cf4-84a9-3a228a676f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.004022 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929686c-6538-4cf4-84a9-3a228a676f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.005713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929686c-6538-4cf4-84a9-3a228a676f09 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.008600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecf90a-f0c7-4d1c-a1ac-6ddbad45588b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.014559 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c1dec-71f3-4915-acc5-bed31a3e9624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.015375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c1dec-71f3-4915-acc5-bed31a3e9624 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.015807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 784/1593] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:59.016529 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c1dec-71f3-4915-acc5-bed31a3e9624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.016653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929686c-6538-4cf4-84a9-3a228a676f09 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.017178 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c1dec-71f3-4915-acc5-bed31a3e9624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.017983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c1dec-71f3-4915-acc5-bed31a3e9624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.020084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c1dec-71f3-4915-acc5-bed31a3e9624 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.028813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929686c-6538-4cf4-84a9-3a228a676f09 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.031974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 810/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:59.032934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c1dec-71f3-4915-acc5-bed31a3e9624 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.036685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d08ee-2a46-43b7-b07f-18196707e777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.037492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d08ee-2a46-43b7-b07f-18196707e777 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.037901 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d08ee-2a46-43b7-b07f-18196707e777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.038235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d08ee-2a46-43b7-b07f-18196707e777 admin admin] PATH_INFO: `/v3/projects/1a4bb3c6a6714a1daae48c35f9b85aba/users/144feda2a2474edfa8dc3e25ae3074b2/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.044053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c1dec-71f3-4915-acc5-bed31a3e9624 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.049163 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd319871-3c18-4240-a776-a0242b5be0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.049806 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd319871-3c18-4240-a776-a0242b5be0da admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.050084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd319871-3c18-4240-a776-a0242b5be0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.050355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd319871-3c18-4240-a776-a0242b5be0da admin admin] PATH_INFO: `/v3/projects/ac5815ead619491680fb44af178e9cc0/users/a397314cdd52409c920071ae009516af/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.054438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d08ee-2a46-43b7-b07f-18196707e777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4bb3c6a6714a1daae48c35f9b85aba, user_id=144feda2a2474edfa8dc3e25ae3074b2, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.057211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d08ee-2a46-43b7-b07f-18196707e777 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.066108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd319871-3c18-4240-a776-a0242b5be0da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac5815ead619491680fb44af178e9cc0, user_id=a397314cdd52409c920071ae009516af, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.067167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-fd319871-3c18-4240-a776-a0242b5be0da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:28:59.067167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 30 19:28:59.067167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: Sep 30 19:28:59.069486 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd319871-3c18-4240-a776-a0242b5be0da admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.071715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 785/1595] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:28:59 2022] PUT /identity/v3/projects/1a4bb3c6a6714a1daae48c35f9b85aba/users/144feda2a2474edfa8dc3e25ae3074b2/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:28:59.072634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d08ee-2a46-43b7-b07f-18196707e777 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.087203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:59.090846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:59.096412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 811/1596] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:28:59 2022] PUT /identity/v3/projects/ac5815ead619491680fb44af178e9cc0/users/a397314cdd52409c920071ae009516af/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:28:59.097788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd319871-3c18-4240-a776-a0242b5be0da admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.143419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d08ee-2a46-43b7-b07f-18196707e777 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.176484 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd319871-3c18-4240-a776-a0242b5be0da admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.183742 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f420e0f-bc45-4b4c-8d96-10b3f89a7a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.184402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f420e0f-bc45-4b4c-8d96-10b3f89a7a50 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.184687 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f420e0f-bc45-4b4c-8d96-10b3f89a7a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.184956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f420e0f-bc45-4b4c-8d96-10b3f89a7a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.185600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f420e0f-bc45-4b4c-8d96-10b3f89a7a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.187200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f420e0f-bc45-4b4c-8d96-10b3f89a7a50 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.199381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 812/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:59.200471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f420e0f-bc45-4b4c-8d96-10b3f89a7a50 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.208470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57808c-66a4-4025-b95a-669706b39fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.209418 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57808c-66a4-4025-b95a-669706b39fde admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.209708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57808c-66a4-4025-b95a-669706b39fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.209970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57808c-66a4-4025-b95a-669706b39fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.210617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57808c-66a4-4025-b95a-669706b39fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.211601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f420e0f-bc45-4b4c-8d96-10b3f89a7a50 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.212449 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57808c-66a4-4025-b95a-669706b39fde admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.216945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b3717-87e6-44c3-9543-1e1060aea954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.217668 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b3717-87e6-44c3-9543-1e1060aea954 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.217974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b3717-87e6-44c3-9543-1e1060aea954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.218310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b3717-87e6-44c3-9543-1e1060aea954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.218953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b3717-87e6-44c3-9543-1e1060aea954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.220967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b3717-87e6-44c3-9543-1e1060aea954 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.224865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 786/1598] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:59.225992 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57808c-66a4-4025-b95a-669706b39fde admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.233250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 813/1599] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:59.234265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b3717-87e6-44c3-9543-1e1060aea954 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.238492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57808c-66a4-4025-b95a-669706b39fde admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.245650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b3717-87e6-44c3-9543-1e1060aea954 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.253340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f3359-84ac-458c-aef7-063b12ec3dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.253509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f3359-84ac-458c-aef7-063b12ec3dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.253768 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f3359-84ac-458c-aef7-063b12ec3dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.254035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f3359-84ac-458c-aef7-063b12ec3dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.254607 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57eea7-f0a3-428d-8c21-2734ffe92dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.254767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f3359-84ac-458c-aef7-063b12ec3dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.256190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57eea7-f0a3-428d-8c21-2734ffe92dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.256969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f3359-84ac-458c-aef7-063b12ec3dc7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.257326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57eea7-f0a3-428d-8c21-2734ffe92dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.257717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57eea7-f0a3-428d-8c21-2734ffe92dfe admin admin] PATH_INFO: `/v3/projects/2f1c4a61fe0e454c805c6e0ed507416f/users/aacda3a1a55046e882e56727033b0d48/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.269965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 814/1600] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:28:59.271324 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54511018-1fc7-45e0-839a-4980ab6dc562 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.271611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54511018-1fc7-45e0-839a-4980ab6dc562 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.271880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54511018-1fc7-45e0-839a-4980ab6dc562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.283344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57eea7-f0a3-428d-8c21-2734ffe92dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1c4a61fe0e454c805c6e0ed507416f, user_id=aacda3a1a55046e882e56727033b0d48, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.286495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57eea7-f0a3-428d-8c21-2734ffe92dfe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.309589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 787/1601] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:28:59 2022] PUT /identity/v3/projects/2f1c4a61fe0e454c805c6e0ed507416f/users/aacda3a1a55046e882e56727033b0d48/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:28:59.311498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec9fd1-950c-40b2-a9e5-e7134d2097c9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.313746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec9fd1-950c-40b2-a9e5-e7134d2097c9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.314593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec9fd1-950c-40b2-a9e5-e7134d2097c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.315925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-54511018-1fc7-45e0-839a-4980ab6dc562 None None] MFA Rules not processed for user `144feda2a2474edfa8dc3e25ae3074b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:59.317313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-54511018-1fc7-45e0-839a-4980ab6dc562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:59.347700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e2ec9fd1-950c-40b2-a9e5-e7134d2097c9 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:59.349051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e2ec9fd1-950c-40b2-a9e5-e7134d2097c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:59.417547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 815/1602] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:59.418292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54511018-1fc7-45e0-839a-4980ab6dc562 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.430088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 788/1603] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:59.430925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec9fd1-950c-40b2-a9e5-e7134d2097c9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.443398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec9fd1-950c-40b2-a9e5-e7134d2097c9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.448352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d50ad-9b37-4b3d-a66d-9a910b285806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.449252 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d50ad-9b37-4b3d-a66d-9a910b285806 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.449632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d50ad-9b37-4b3d-a66d-9a910b285806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.450052 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d50ad-9b37-4b3d-a66d-9a910b285806 admin admin] PATH_INFO: `/v3/projects/d9f37085e1ca47a88dc01edd217a9805/users/4427e3b5656a407dbbf7608985a64ab8/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.468061 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d50ad-9b37-4b3d-a66d-9a910b285806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f37085e1ca47a88dc01edd217a9805, user_id=4427e3b5656a407dbbf7608985a64ab8, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.471806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d50ad-9b37-4b3d-a66d-9a910b285806 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.495853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 789/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:28:59 2022] PUT /identity/v3/projects/d9f37085e1ca47a88dc01edd217a9805/users/4427e3b5656a407dbbf7608985a64ab8/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:28:59.497144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d50ad-9b37-4b3d-a66d-9a910b285806 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.519433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:59.552110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54511018-1fc7-45e0-839a-4980ab6dc562 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.561216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:59.599485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d50ad-9b37-4b3d-a66d-9a910b285806 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.606732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0525fa3-43f8-4dcd-8bef-cce96661ef86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.607579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0525fa3-43f8-4dcd-8bef-cce96661ef86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.607890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0525fa3-43f8-4dcd-8bef-cce96661ef86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.608268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0525fa3-43f8-4dcd-8bef-cce96661ef86 admin admin] PATH_INFO: `/v3/projects/395bb9b09f7948bbad5817c88aa4accc/users/f7aa40f3966d4c97ab6a1a764722969d/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.631426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7409ba-a6ce-4e78-97b1-6a15748cd543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.632309 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7409ba-a6ce-4e78-97b1-6a15748cd543 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.632627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7409ba-a6ce-4e78-97b1-6a15748cd543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.632907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7409ba-a6ce-4e78-97b1-6a15748cd543 admin admin] PATH_INFO: `/v3/projects/de44d50bb79d48189cd74cad8079e01f/users/624738daa9db4aff885d71ea2d65fb26/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.634084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0525fa3-43f8-4dcd-8bef-cce96661ef86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=395bb9b09f7948bbad5817c88aa4accc, user_id=f7aa40f3966d4c97ab6a1a764722969d, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.636953 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0525fa3-43f8-4dcd-8bef-cce96661ef86 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.653469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 790/1605] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:28:59 2022] PUT /identity/v3/projects/395bb9b09f7948bbad5817c88aa4accc/users/f7aa40f3966d4c97ab6a1a764722969d/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:28:59.654570 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0525fa3-43f8-4dcd-8bef-cce96661ef86 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.661276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7409ba-a6ce-4e78-97b1-6a15748cd543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de44d50bb79d48189cd74cad8079e01f, user_id=624738daa9db4aff885d71ea2d65fb26, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.664511 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7409ba-a6ce-4e78-97b1-6a15748cd543 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.684325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 816/1606] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:28:59 2022] PUT /identity/v3/projects/de44d50bb79d48189cd74cad8079e01f/users/624738daa9db4aff885d71ea2d65fb26/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:28:59.686552 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a77a05-0553-4d14-875b-bdacb3b549a1 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.686552 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a77a05-0553-4d14-875b-bdacb3b549a1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.686552 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a77a05-0553-4d14-875b-bdacb3b549a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.728892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0525fa3-43f8-4dcd-8bef-cce96661ef86 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.731448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-75a77a05-0553-4d14-875b-bdacb3b549a1 None None] MFA Rules not processed for user `aacda3a1a55046e882e56727033b0d48`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:59.732811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-75a77a05-0553-4d14-875b-bdacb3b549a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:59.803805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c2e30-1c00-4c14-a0a5-f4af55d955d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:28:59.804665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c2e30-1c00-4c14-a0a5-f4af55d955d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.805041 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c2e30-1c00-4c14-a0a5-f4af55d955d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.805434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c2e30-1c00-4c14-a0a5-f4af55d955d6 admin admin] PATH_INFO: `/v3/projects/612fb8a339e4478ba78bc5ef3b343b82/users/ccbdadc310fc4bf8a6679bcb39b34ff1/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.819909 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 817/1607] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:28:59.820929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a77a05-0553-4d14-875b-bdacb3b549a1 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:28:59.824347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-75a77a05-0553-4d14-875b-bdacb3b549a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:59.836521 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c2e30-1c00-4c14-a0a5-f4af55d955d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=612fb8a339e4478ba78bc5ef3b343b82, user_id=ccbdadc310fc4bf8a6679bcb39b34ff1, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:28:59.840341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c2e30-1c00-4c14-a0a5-f4af55d955d6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:28:59.864838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 791/1608] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:28:59 2022] PUT /identity/v3/projects/612fb8a339e4478ba78bc5ef3b343b82/users/ccbdadc310fc4bf8a6679bcb39b34ff1/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:28:59.867255 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a2c89-ff58-4414-9f75-a068bd8d25f9 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.867550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a2c89-ff58-4414-9f75-a068bd8d25f9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.867849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a2c89-ff58-4414-9f75-a068bd8d25f9 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.877459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 792/1609] 10.222.0.17 () {54 vars in 961 bytes} [Fri Sep 30 19:28:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:28:59.879864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b658c27-851a-4bb6-bb6b-5cd691b2b8a5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:28:59.880269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b658c27-851a-4bb6-bb6b-5cd691b2b8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:28:59.881294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b658c27-851a-4bb6-bb6b-5cd691b2b8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:28:59.917876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a77a05-0553-4d14-875b-bdacb3b549a1 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:28:59.921749 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4b658c27-851a-4bb6-bb6b-5cd691b2b8a5 None None] MFA Rules not processed for user `4427e3b5656a407dbbf7608985a64ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:28:59.923182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4b658c27-851a-4bb6-bb6b-5cd691b2b8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:28:59.939864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:28:59.977969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:00.008422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bda3f-591b-41e4-bb1d-699b384be2e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:00.009241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bda3f-591b-41e4-bb1d-699b384be2e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.009614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bda3f-591b-41e4-bb1d-699b384be2e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.010322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bda3f-591b-41e4-bb1d-699b384be2e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.028669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 793/1610] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:28:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:00.028669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f4e1a-3e86-4260-951f-dfec9d845f6e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.028669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f4e1a-3e86-4260-951f-dfec9d845f6e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.028669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f4e1a-3e86-4260-951f-dfec9d845f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.053965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-515f4e1a-3e86-4260-951f-dfec9d845f6e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:00.055862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-515f4e1a-3e86-4260-951f-dfec9d845f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:00.108549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bda3f-591b-41e4-bb1d-699b384be2e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:00.109941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bda3f-591b-41e4-bb1d-699b384be2e8 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:00.138010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 818/1611] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:00.141074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb980-3706-4494-8ea3-19bf37c7eda0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.141382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb980-3706-4494-8ea3-19bf37c7eda0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.141651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb980-3706-4494-8ea3-19bf37c7eda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.154237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 794/1612] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:00.155135 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f4e1a-3e86-4260-951f-dfec9d845f6e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:00.175060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-13cbb980-3706-4494-8ea3-19bf37c7eda0 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:00.176849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-13cbb980-3706-4494-8ea3-19bf37c7eda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:00.193387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 819/1613] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:00.196895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25a0a9-ff46-4fba-bdb3-183ad8914449 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.197362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25a0a9-ff46-4fba-bdb3-183ad8914449 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.198192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25a0a9-ff46-4fba-bdb3-183ad8914449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.228414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-db25a0a9-ff46-4fba-bdb3-183ad8914449 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:00.230172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-db25a0a9-ff46-4fba-bdb3-183ad8914449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:00.236613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f4e1a-3e86-4260-951f-dfec9d845f6e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:00.241446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953bfa0-c317-48ad-90d5-fff71a139268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:00.242076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953bfa0-c317-48ad-90d5-fff71a139268 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.242362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953bfa0-c317-48ad-90d5-fff71a139268 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.242632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953bfa0-c317-48ad-90d5-fff71a139268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.246965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 820/1614] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:00.248789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba38e06-91df-4991-af18-38049b123782 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.249222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba38e06-91df-4991-af18-38049b123782 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.249788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba38e06-91df-4991-af18-38049b123782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.319626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-eba38e06-91df-4991-af18-38049b123782 None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:00.323538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-eba38e06-91df-4991-af18-38049b123782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:00.346238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953bfa0-c317-48ad-90d5-fff71a139268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:00.347877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953bfa0-c317-48ad-90d5-fff71a139268 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:00.360421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 795/1615] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:00.361745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f1173-98f0-4abb-a0e5-072d1e3af5d5 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.362841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f1173-98f0-4abb-a0e5-072d1e3af5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.363213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f1173-98f0-4abb-a0e5-072d1e3af5d5 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.370709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 796/1616] 10.222.0.17 () {54 vars in 961 bytes} [Fri Sep 30 19:29:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:29:00.371760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f1173-98f0-4abb-a0e5-072d1e3af5d5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:00.391025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f1173-98f0-4abb-a0e5-072d1e3af5d5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:00.394972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345e487-0add-49f1-baff-06962a96b2ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:00.396566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345e487-0add-49f1-baff-06962a96b2ba service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.396566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345e487-0add-49f1-baff-06962a96b2ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.396566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345e487-0add-49f1-baff-06962a96b2ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.407078 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:00.414143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345e487-0add-49f1-baff-06962a96b2ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:00.417698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345e487-0add-49f1-baff-06962a96b2ba service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:00.504550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 797/1617] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:00.504550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345e487-0add-49f1-baff-06962a96b2ba service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:00.504550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345e487-0add-49f1-baff-06962a96b2ba service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:00.504550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176a0a2-9254-46d1-abbb-c11dab9308c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:00.506493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176a0a2-9254-46d1-abbb-c11dab9308c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.506493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176a0a2-9254-46d1-abbb-c11dab9308c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.506493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176a0a2-9254-46d1-abbb-c11dab9308c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.510347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 821/1618] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Sep 30 19:29:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:00.527562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176a0a2-9254-46d1-abbb-c11dab9308c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:00.532821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176a0a2-9254-46d1-abbb-c11dab9308c3 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:00.533019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba38e06-91df-4991-af18-38049b123782 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:00.550883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba38e06-91df-4991-af18-38049b123782 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:00.556091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 798/1619] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:00.560216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5da969-e1d0-492b-a5c0-709f9454bd87 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.560564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5da969-e1d0-492b-a5c0-709f9454bd87 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.560861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5da969-e1d0-492b-a5c0-709f9454bd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.562301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9823c5-829e-4703-9829-c1b5a1c2deb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:00.562982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9823c5-829e-4703-9829-c1b5a1c2deb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.563288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9823c5-829e-4703-9829-c1b5a1c2deb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.563571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9823c5-829e-4703-9829-c1b5a1c2deb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.589563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9823c5-829e-4703-9829-c1b5a1c2deb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:00.591146 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9823c5-829e-4703-9829-c1b5a1c2deb2 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:00.606785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3d5da969-e1d0-492b-a5c0-709f9454bd87 None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:00.608025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3d5da969-e1d0-492b-a5c0-709f9454bd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:00.611053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 822/1620] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:00.611914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9823c5-829e-4703-9829-c1b5a1c2deb2 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:00.634867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 799/1621] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Sep 30 19:29:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:00.644931 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5da969-e1d0-492b-a5c0-709f9454bd87 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:00.656589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9823c5-829e-4703-9829-c1b5a1c2deb2 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:00.664807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5da969-e1d0-492b-a5c0-709f9454bd87 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:00.673498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e00349-8988-431e-9533-3938692cb3ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:00.674182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e00349-8988-431e-9533-3938692cb3ca service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.674638 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e00349-8988-431e-9533-3938692cb3ca service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.674913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e00349-8988-431e-9533-3938692cb3ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.694981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8322272-dd1d-4dfb-82a6-8a922041d08f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:00.696107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8322272-dd1d-4dfb-82a6-8a922041d08f service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:00.696634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8322272-dd1d-4dfb-82a6-8a922041d08f service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:00.697095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8322272-dd1d-4dfb-82a6-8a922041d08f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:00.730627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8322272-dd1d-4dfb-82a6-8a922041d08f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:00.740816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8322272-dd1d-4dfb-82a6-8a922041d08f service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:00.789459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 800/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:00.834207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:00.877493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e00349-8988-431e-9533-3938692cb3ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:00.879729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e00349-8988-431e-9533-3938692cb3ca service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:00.907604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 823/1623] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:01.114412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912764d8-8ceb-4790-abc4-6d70136c49f8 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:01.115438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912764d8-8ceb-4790-abc4-6d70136c49f8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:01.115833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912764d8-8ceb-4790-abc4-6d70136c49f8 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:01.134279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 801/1624] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 30 19:29:01 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:29:01.140297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cd2aa-ccd1-41d7-ba10-7f5704d6101f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:01.140297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cd2aa-ccd1-41d7-ba10-7f5704d6101f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:01.140914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cd2aa-ccd1-41d7-ba10-7f5704d6101f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:01.190257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a74cd2aa-ccd1-41d7-ba10-7f5704d6101f None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:01.197311 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a74cd2aa-ccd1-41d7-ba10-7f5704d6101f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:01.223820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 824/1625] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:29:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:01.234608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912764d8-8ceb-4790-abc4-6d70136c49f8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:01.262992 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912764d8-8ceb-4790-abc4-6d70136c49f8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:01.271682 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29247f5f-3037-4866-aca2-81db90520cf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:01.272320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29247f5f-3037-4866-aca2-81db90520cf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:01.272730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29247f5f-3037-4866-aca2-81db90520cf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:01.272871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29247f5f-3037-4866-aca2-81db90520cf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:01.298140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29247f5f-3037-4866-aca2-81db90520cf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:01.299575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29247f5f-3037-4866-aca2-81db90520cf9 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:01.323792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 802/1626] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:01.618176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8755af8-de75-4bd4-ae8b-45a1ae81db94 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:01.619065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8755af8-de75-4bd4-ae8b-45a1ae81db94 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:01.619650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8755af8-de75-4bd4-ae8b-45a1ae81db94 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:01.622252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 825/1627] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 30 19:29:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:29:01.630019 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790049d-22b0-4e68-ba22-224e5c4aaa28 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:01.631237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790049d-22b0-4e68-ba22-224e5c4aaa28 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:01.631895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790049d-22b0-4e68-ba22-224e5c4aaa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:01.692744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3790049d-22b0-4e68-ba22-224e5c4aaa28 None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:01.696072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3790049d-22b0-4e68-ba22-224e5c4aaa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:01.721022 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 803/1628] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:29:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:01.735115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8755af8-de75-4bd4-ae8b-45a1ae81db94 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:01.762786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8755af8-de75-4bd4-ae8b-45a1ae81db94 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:01.770832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb1e90-2203-4d33-8a2a-e05f1e182d17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:01.771701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1e90-2203-4d33-8a2a-e05f1e182d17 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:01.772487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1e90-2203-4d33-8a2a-e05f1e182d17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:01.772770 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1e90-2203-4d33-8a2a-e05f1e182d17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:01.806061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb1e90-2203-4d33-8a2a-e05f1e182d17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:01.811216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb1e90-2203-4d33-8a2a-e05f1e182d17 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:01.842893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 826/1629] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:02.190379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790049d-22b0-4e68-ba22-224e5c4aaa28 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:02.218870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790049d-22b0-4e68-ba22-224e5c4aaa28 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:02.235588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:02.237692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:02.238270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:02.238853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] PATH_INFO: `/v3/users/144feda2a2474edfa8dc3e25ae3074b2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:02.253267 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=144feda2a2474edfa8dc3e25ae3074b2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:02.256640 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:02.414747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '144feda2a2474edfa8dc3e25ae3074b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:02.431848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '144feda2a2474edfa8dc3e25ae3074b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:02.432831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '144feda2a2474edfa8dc3e25ae3074b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:02.443317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '144feda2a2474edfa8dc3e25ae3074b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:02.480551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '144feda2a2474edfa8dc3e25ae3074b2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:02.491563 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 804/1630] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:02 2022] DELETE /identity/v3/users/144feda2a2474edfa8dc3e25ae3074b2 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:02.493570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2033-8878-4040-b159-aa2576933ec7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:02.494436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2033-8878-4040-b159-aa2576933ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:02.494901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2033-8878-4040-b159-aa2576933ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:02.520334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ed1f2033-8878-4040-b159-aa2576933ec7 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:02.522057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ed1f2033-8878-4040-b159-aa2576933ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:02.656044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 827/1631] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:02.669599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:02.672447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:02.819962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6a438-2aa4-4c78-9918-6d7ae5be28fc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:02.825484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d4978-1cce-464e-96d3-a74677cd59eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:02.826251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d4978-1cce-464e-96d3-a74677cd59eb service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:02.827277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d4978-1cce-464e-96d3-a74677cd59eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:02.827801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d4978-1cce-464e-96d3-a74677cd59eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:02.842628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d4978-1cce-464e-96d3-a74677cd59eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:02.843978 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d4978-1cce-464e-96d3-a74677cd59eb service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:02.872117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 805/1632] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:05.242582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f2033-8878-4040-b159-aa2576933ec7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:05.244321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ed1f2033-8878-4040-b159-aa2576933ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:05.266707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f2033-8878-4040-b159-aa2576933ec7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:05.271931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:05.273455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:05.273895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:05.274334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] PATH_INFO: `/v3/projects/1a4bb3c6a6714a1daae48c35f9b85aba` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:05.276548 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a4bb3c6a6714a1daae48c35f9b85aba)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:05.280295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:05.446601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a4bb3c6a6714a1daae48c35f9b85aba', 'request_id': 'req-b69b11a5-f6ef-47bb-9457-d194f8709a1a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:05.453675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a4bb3c6a6714a1daae48c35f9b85aba', 'request_id': 'req-b69b11a5-f6ef-47bb-9457-d194f8709a1a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:05.468253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a4bb3c6a6714a1daae48c35f9b85aba', 'request_id': 'req-b69b11a5-f6ef-47bb-9457-d194f8709a1a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:05.488424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] The token cache is being invalidate because project 1a4bb3c6a6714a1daae48c35f9b85aba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:05.488967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:05.490523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b69b11a5-f6ef-47bb-9457-d194f8709a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:05.493254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 828/1633] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:05 2022] DELETE /identity/v3/projects/1a4bb3c6a6714a1daae48c35f9b85aba => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:29:05.498312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42358b6a-5124-402b-a5ab-a4ab64ae36ba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:05.498895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42358b6a-5124-402b-a5ab-a4ab64ae36ba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:05.499183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42358b6a-5124-402b-a5ab-a4ab64ae36ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:05.535627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-42358b6a-5124-402b-a5ab-a4ab64ae36ba None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:05.536954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-42358b6a-5124-402b-a5ab-a4ab64ae36ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:05.582266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:05.705292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d6900-b361-4e37-b0b0-52821a7f6c37 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:05.705700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d6900-b361-4e37-b0b0-52821a7f6c37 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:05.706044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d6900-b361-4e37-b0b0-52821a7f6c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:05.714908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 806/1634] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:05.742643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-dd7d6900-b361-4e37-b0b0-52821a7f6c37 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:05.746994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dd7d6900-b361-4e37-b0b0-52821a7f6c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:05.761227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42358b6a-5124-402b-a5ab-a4ab64ae36ba None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:05.767831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 829/1635] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:29:05.774006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42358b6a-5124-402b-a5ab-a4ab64ae36ba None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:05.780090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263d3fd-c4d9-4cc3-96cf-74497a3a1470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:05.781083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263d3fd-c4d9-4cc3-96cf-74497a3a1470 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:05.781581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263d3fd-c4d9-4cc3-96cf-74497a3a1470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:05.782046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263d3fd-c4d9-4cc3-96cf-74497a3a1470 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:05.782904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263d3fd-c4d9-4cc3-96cf-74497a3a1470 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:29:05.783424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263d3fd-c4d9-4cc3-96cf-74497a3a1470 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:05.786016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263d3fd-c4d9-4cc3-96cf-74497a3a1470 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:05.809471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 807/1636] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:29:05.813731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d6900-b361-4e37-b0b0-52821a7f6c37 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:05.827347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d6900-b361-4e37-b0b0-52821a7f6c37 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:05.832649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75c980-105e-4f78-b19a-bc52c14c2f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:05.833476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75c980-105e-4f78-b19a-bc52c14c2f08 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:05.833871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75c980-105e-4f78-b19a-bc52c14c2f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:05.834258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75c980-105e-4f78-b19a-bc52c14c2f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:05.835001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75c980-105e-4f78-b19a-bc52c14c2f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:05.837539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75c980-105e-4f78-b19a-bc52c14c2f08 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:05.883276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 830/1637] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:05 2022] POST /identity/v3/projects => generated 405 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:05.888515 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263d3fd-c4d9-4cc3-96cf-74497a3a1470 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:05.903624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263d3fd-c4d9-4cc3-96cf-74497a3a1470 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:06.000608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628485e-d751-4e0d-a825-2c29b5d7c216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:06.001516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628485e-d751-4e0d-a825-2c29b5d7c216 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:06.001516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628485e-d751-4e0d-a825-2c29b5d7c216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:06.001954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628485e-d751-4e0d-a825-2c29b5d7c216 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:06.005938 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7628485e-d751-4e0d-a825-2c29b5d7c216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:06.010040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7628485e-d751-4e0d-a825-2c29b5d7c216 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:06.047220 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 808/1638] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:05 2022] POST /identity/v3/users => generated 303 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:06.051615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75c980-105e-4f78-b19a-bc52c14c2f08 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:06.063407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75c980-105e-4f78-b19a-bc52c14c2f08 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:06.074119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f587f7-f19c-406c-b0d3-53d1abe8a211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:06.075924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f587f7-f19c-406c-b0d3-53d1abe8a211 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:06.077690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f587f7-f19c-406c-b0d3-53d1abe8a211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:06.078597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f587f7-f19c-406c-b0d3-53d1abe8a211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:06.078857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f587f7-f19c-406c-b0d3-53d1abe8a211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:06.081542 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f587f7-f19c-406c-b0d3-53d1abe8a211 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:06.104895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 831/1639] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:06.108911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628485e-d751-4e0d-a825-2c29b5d7c216 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:06.123115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628485e-d751-4e0d-a825-2c29b5d7c216 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:06.134598 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba988ae-25cf-44bb-90f2-34cd68e0d249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:06.135901 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba988ae-25cf-44bb-90f2-34cd68e0d249 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:06.136202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba988ae-25cf-44bb-90f2-34cd68e0d249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:06.136502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba988ae-25cf-44bb-90f2-34cd68e0d249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:06.137154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba988ae-25cf-44bb-90f2-34cd68e0d249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:06.138782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba988ae-25cf-44bb-90f2-34cd68e0d249 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:06.156866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 809/1640] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:06.157556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f587f7-f19c-406c-b0d3-53d1abe8a211 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:06.168338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:06.173805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f587f7-f19c-406c-b0d3-53d1abe8a211 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:06.181153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b856297-f65c-4302-93cb-30e5e54e37af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:06.182536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b856297-f65c-4302-93cb-30e5e54e37af admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:06.183063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b856297-f65c-4302-93cb-30e5e54e37af admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:06.183659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b856297-f65c-4302-93cb-30e5e54e37af admin admin] PATH_INFO: `/v3/projects/92c67b8aba4c4a688e8d8c6f6c7caf24/users/54570d9bcfcf4800bcd8135fdb579ae8/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:06.223539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b856297-f65c-4302-93cb-30e5e54e37af admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c67b8aba4c4a688e8d8c6f6c7caf24, user_id=54570d9bcfcf4800bcd8135fdb579ae8, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:06.227383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b856297-f65c-4302-93cb-30e5e54e37af admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:06.252623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 832/1641] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:06 2022] PUT /identity/v3/projects/92c67b8aba4c4a688e8d8c6f6c7caf24/users/54570d9bcfcf4800bcd8135fdb579ae8/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:06.261963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a89e8-a149-4046-af0f-5f5ab6f72f45 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:06.262326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a89e8-a149-4046-af0f-5f5ab6f72f45 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:06.263020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a89e8-a149-4046-af0f-5f5ab6f72f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:06.314194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-506a89e8-a149-4046-af0f-5f5ab6f72f45 None None] MFA Rules not processed for user `54570d9bcfcf4800bcd8135fdb579ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:06.316859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-506a89e8-a149-4046-af0f-5f5ab6f72f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:06.453813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 810/1642] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:06 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:29:06.462587 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b856297-f65c-4302-93cb-30e5e54e37af admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:06.591898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b856297-f65c-4302-93cb-30e5e54e37af admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:06.597586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68da50-84b2-4f9a-be88-e0e08734952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:06.598302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68da50-84b2-4f9a-be88-e0e08734952d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:06.598641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68da50-84b2-4f9a-be88-e0e08734952d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:06.599935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68da50-84b2-4f9a-be88-e0e08734952d admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:06.600712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68da50-84b2-4f9a-be88-e0e08734952d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:06.612762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68da50-84b2-4f9a-be88-e0e08734952d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:06.706876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 833/1643] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:06 2022] POST /identity/v3/projects => generated 405 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:06.710852 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a89e8-a149-4046-af0f-5f5ab6f72f45 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:06.729872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a89e8-a149-4046-af0f-5f5ab6f72f45 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:06.783561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:06.930790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbde59-59a2-4364-8868-5d52bb80e70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:06.933045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbde59-59a2-4364-8868-5d52bb80e70f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:06.935126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbde59-59a2-4364-8868-5d52bb80e70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:06.935855 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbde59-59a2-4364-8868-5d52bb80e70f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:06.938381 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbde59-59a2-4364-8868-5d52bb80e70f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:06.940867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbde59-59a2-4364-8868-5d52bb80e70f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:06.995549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 811/1644] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:06 2022] POST /identity/v3/users => generated 309 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:06.999891 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68da50-84b2-4f9a-be88-e0e08734952d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:07.015060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68da50-84b2-4f9a-be88-e0e08734952d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:07.028711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba26e0-4be1-46b5-8e19-f45c6fb555df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:07.029558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba26e0-4be1-46b5-8e19-f45c6fb555df admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:07.029638 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba26e0-4be1-46b5-8e19-f45c6fb555df admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:07.029874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba26e0-4be1-46b5-8e19-f45c6fb555df admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:07.030521 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba26e0-4be1-46b5-8e19-f45c6fb555df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:07.043503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba26e0-4be1-46b5-8e19-f45c6fb555df admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:07.076433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 834/1645] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:07.080495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbde59-59a2-4364-8868-5d52bb80e70f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:07.095482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbde59-59a2-4364-8868-5d52bb80e70f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:07.101470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2ac88-b633-4a10-8a18-0c6d5f0a45e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:07.102431 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2ac88-b633-4a10-8a18-0c6d5f0a45e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:07.102871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2ac88-b633-4a10-8a18-0c6d5f0a45e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:07.104167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2ac88-b633-4a10-8a18-0c6d5f0a45e6 admin admin] PATH_INFO: `/v3/projects/1890a51f09c146349299ed243a284480/users/884fb021387e484fb2033e764565cdc8/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:07.127608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2ac88-b633-4a10-8a18-0c6d5f0a45e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1890a51f09c146349299ed243a284480, user_id=884fb021387e484fb2033e764565cdc8, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:07.130879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2ac88-b633-4a10-8a18-0c6d5f0a45e6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:07.155282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 812/1646] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:07 2022] PUT /identity/v3/projects/1890a51f09c146349299ed243a284480/users/884fb021387e484fb2033e764565cdc8/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:07.159839 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba26e0-4be1-46b5-8e19-f45c6fb555df admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:07.174778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba26e0-4be1-46b5-8e19-f45c6fb555df admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:07.280792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d43d4d-22b8-46a1-b06c-df2dece03852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:07.282138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d43d4d-22b8-46a1-b06c-df2dece03852 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:07.283128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d43d4d-22b8-46a1-b06c-df2dece03852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:07.284099 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d43d4d-22b8-46a1-b06c-df2dece03852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:07.285253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d43d4d-22b8-46a1-b06c-df2dece03852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:07.287272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d43d4d-22b8-46a1-b06c-df2dece03852 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:07.351546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2ac88-b633-4a10-8a18-0c6d5f0a45e6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:07.362916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 835/1647] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:07.368657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2ac88-b633-4a10-8a18-0c6d5f0a45e6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:07.386537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba40d0-c741-44ec-929a-3852eeb4d61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:07.394533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba40d0-c741-44ec-929a-3852eeb4d61f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:07.395174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba40d0-c741-44ec-929a-3852eeb4d61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:07.396637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba40d0-c741-44ec-929a-3852eeb4d61f admin admin] PATH_INFO: `/v3/projects/1890a51f09c146349299ed243a284480/users/884fb021387e484fb2033e764565cdc8/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:07.409791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba40d0-c741-44ec-929a-3852eeb4d61f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1890a51f09c146349299ed243a284480, user_id=884fb021387e484fb2033e764565cdc8, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:07.414643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba40d0-c741-44ec-929a-3852eeb4d61f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:07.435558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 813/1648] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:07 2022] PUT /identity/v3/projects/1890a51f09c146349299ed243a284480/users/884fb021387e484fb2033e764565cdc8/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:07.446040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c266a-3022-4a10-b393-c8562e32ce76 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:07.446469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c266a-3022-4a10-b393-c8562e32ce76 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:07.446852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c266a-3022-4a10-b393-c8562e32ce76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:07.497635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-490c266a-3022-4a10-b393-c8562e32ce76 None None] MFA Rules not processed for user `884fb021387e484fb2033e764565cdc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:07.499880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-490c266a-3022-4a10-b393-c8562e32ce76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:07.615186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:07.693670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 836/1649] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:07 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:07.702138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba40d0-c741-44ec-929a-3852eeb4d61f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:07.704110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-23ba40d0-c741-44ec-929a-3852eeb4d61f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:07.840470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba40d0-c741-44ec-929a-3852eeb4d61f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:07.846733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139116e5-d372-49c1-ac6a-ce036e457011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:07.847377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139116e5-d372-49c1-ac6a-ce036e457011 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:07.847649 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139116e5-d372-49c1-ac6a-ce036e457011 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:07.847915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139116e5-d372-49c1-ac6a-ce036e457011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:07.871174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139116e5-d372-49c1-ac6a-ce036e457011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:07.872970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139116e5-d372-49c1-ac6a-ce036e457011 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:07.900524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 814/1650] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:07 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:10.542210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c266a-3022-4a10-b393-c8562e32ce76 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:10.695947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c266a-3022-4a10-b393-c8562e32ce76 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:10.713145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:10.714049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:10.714467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:10.715069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] PATH_INFO: `/v3/users/54570d9bcfcf4800bcd8135fdb579ae8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:10.720361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54570d9bcfcf4800bcd8135fdb579ae8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:10.721437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: WARNING py.warnings [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:29:10.721437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: warnings.warn(msg) Sep 30 19:29:10.721437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: Sep 30 19:29:10.722663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:10.920941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54570d9bcfcf4800bcd8135fdb579ae8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:10.961535 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54570d9bcfcf4800bcd8135fdb579ae8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:10.962135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54570d9bcfcf4800bcd8135fdb579ae8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:10.971764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54570d9bcfcf4800bcd8135fdb579ae8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:11.033113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a44073ca-cf36-41b9-86bc-e5eba2115de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54570d9bcfcf4800bcd8135fdb579ae8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:11.050631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 837/1651] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:10 2022] DELETE /identity/v3/users/54570d9bcfcf4800bcd8135fdb579ae8 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:11.054911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08721311-3652-48c8-a3a9-b54d6d97e8ca None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:11.055750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08721311-3652-48c8-a3a9-b54d6d97e8ca None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:11.056163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08721311-3652-48c8-a3a9-b54d6d97e8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:11.100553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc281b4-e1ff-41f4-8c6f-bb64a64ee078 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:11.100949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc281b4-e1ff-41f4-8c6f-bb64a64ee078 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:11.101324 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc281b4-e1ff-41f4-8c6f-bb64a64ee078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:11.157801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0fc281b4-e1ff-41f4-8c6f-bb64a64ee078 None None] MFA Rules not processed for user `f1bb718f9153493ca6dda343d461c5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:11.159492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0fc281b4-e1ff-41f4-8c6f-bb64a64ee078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:11.168380 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-08721311-3652-48c8-a3a9-b54d6d97e8ca None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:11.169569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-08721311-3652-48c8-a3a9-b54d6d97e8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:11.265878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:11.310480 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 838/1652] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:11.323302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae8723-2726-4a7e-8a3e-95da96af41a7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:11.323591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae8723-2726-4a7e-8a3e-95da96af41a7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:11.323868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae8723-2726-4a7e-8a3e-95da96af41a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:11.348456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 815/1653] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:29:11.369447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e8ae8723-2726-4a7e-8a3e-95da96af41a7 None None] MFA Rules not processed for user `f1bb718f9153493ca6dda343d461c5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:11.375797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e8ae8723-2726-4a7e-8a3e-95da96af41a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:11.397627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 839/1654] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:11.413643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08721311-3652-48c8-a3a9-b54d6d97e8ca None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:11.416541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-08721311-3652-48c8-a3a9-b54d6d97e8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:11.435381 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:11.459849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae8723-2726-4a7e-8a3e-95da96af41a7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:11.460555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e8ae8723-2726-4a7e-8a3e-95da96af41a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:11.631958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae8723-2726-4a7e-8a3e-95da96af41a7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:11.643837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62916746-62f7-45da-b82d-ffe32f0da2b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:11.644716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62916746-62f7-45da-b82d-ffe32f0da2b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:11.645168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62916746-62f7-45da-b82d-ffe32f0da2b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:11.645752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62916746-62f7-45da-b82d-ffe32f0da2b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:11.677675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62916746-62f7-45da-b82d-ffe32f0da2b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:11.680409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62916746-62f7-45da-b82d-ffe32f0da2b8 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:11.691086 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08721311-3652-48c8-a3a9-b54d6d97e8ca None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:11.703472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee1b32-ecb7-465a-b6f2-87273e55e33f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:11.706225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee1b32-ecb7-465a-b6f2-87273e55e33f service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:11.706830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee1b32-ecb7-465a-b6f2-87273e55e33f service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:11.707114 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee1b32-ecb7-465a-b6f2-87273e55e33f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:11.720588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 840/1655] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:11.730939 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee1b32-ecb7-465a-b6f2-87273e55e33f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:11.732588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee1b32-ecb7-465a-b6f2-87273e55e33f service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:11.802505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 816/1656] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:14.279369 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70905-d8b3-4db2-a216-9ff033f98b4d None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:14.279911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70905-d8b3-4db2-a216-9ff033f98b4d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:14.279911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70905-d8b3-4db2-a216-9ff033f98b4d None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:14.282507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 841/1657] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 30 19:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:29:14.295592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad006b-5175-4b56-ab31-99ecd4d18d37 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:14.296425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad006b-5175-4b56-ab31-99ecd4d18d37 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:14.296878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad006b-5175-4b56-ab31-99ecd4d18d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:14.433822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0aad006b-5175-4b56-ab31-99ecd4d18d37 None None] MFA Rules not processed for user `18ea49d1f6c040f18511535b7e51a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:14.435162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0aad006b-5175-4b56-ab31-99ecd4d18d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:14.457060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 817/1658] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:29:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:29:14.532505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70905-d8b3-4db2-a216-9ff033f98b4d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:14.571853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70905-d8b3-4db2-a216-9ff033f98b4d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:14.588910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507fd11-4690-49dc-963b-e803bfc11324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:14.589617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507fd11-4690-49dc-963b-e803bfc11324 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:14.589898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507fd11-4690-49dc-963b-e803bfc11324 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:14.590169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507fd11-4690-49dc-963b-e803bfc11324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:14.616004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507fd11-4690-49dc-963b-e803bfc11324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:14.617673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507fd11-4690-49dc-963b-e803bfc11324 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:14.643359 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 842/1659] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:15.233855 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad006b-5175-4b56-ab31-99ecd4d18d37 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:15.234692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0aad006b-5175-4b56-ab31-99ecd4d18d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:15.265070 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad006b-5175-4b56-ab31-99ecd4d18d37 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:15.275318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe1f03-d3f2-4e5e-92b7-a701be74ca17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:15.275973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe1f03-d3f2-4e5e-92b7-a701be74ca17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:15.276251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe1f03-d3f2-4e5e-92b7-a701be74ca17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:15.276549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe1f03-d3f2-4e5e-92b7-a701be74ca17 admin admin] PATH_INFO: `/v3/users/4427e3b5656a407dbbf7608985a64ab8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:15.282331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe1f03-d3f2-4e5e-92b7-a701be74ca17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4427e3b5656a407dbbf7608985a64ab8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:15.289867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe1f03-d3f2-4e5e-92b7-a701be74ca17 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:15.386426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcfacc-7940-4b1e-93ff-426cffb89bf5 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:15.387546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcfacc-7940-4b1e-93ff-426cffb89bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:15.387989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcfacc-7940-4b1e-93ff-426cffb89bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:15.407594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dffe1f03-d3f2-4e5e-92b7-a701be74ca17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4427e3b5656a407dbbf7608985a64ab8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:15.428921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dffe1f03-d3f2-4e5e-92b7-a701be74ca17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4427e3b5656a407dbbf7608985a64ab8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:15.432283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dffe1f03-d3f2-4e5e-92b7-a701be74ca17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4427e3b5656a407dbbf7608985a64ab8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:15.447279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dffe1f03-d3f2-4e5e-92b7-a701be74ca17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4427e3b5656a407dbbf7608985a64ab8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:15.475562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-dffe1f03-d3f2-4e5e-92b7-a701be74ca17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4427e3b5656a407dbbf7608985a64ab8'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:15.480524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4bbcfacc-7940-4b1e-93ff-426cffb89bf5 None None] MFA Rules not processed for user `a397314cdd52409c920071ae009516af`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:15.481834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4bbcfacc-7940-4b1e-93ff-426cffb89bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:15.489429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 818/1660] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:15 2022] DELETE /identity/v3/users/4427e3b5656a407dbbf7608985a64ab8 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:15.501808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094dc28-0632-4384-9c6e-f2e830b1668d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:15.502492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094dc28-0632-4384-9c6e-f2e830b1668d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:15.517564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094dc28-0632-4384-9c6e-f2e830b1668d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:15.608509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6094dc28-0632-4384-9c6e-f2e830b1668d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:15.621287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6094dc28-0632-4384-9c6e-f2e830b1668d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:15.664618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 843/1661] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:15.667934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcfacc-7940-4b1e-93ff-426cffb89bf5 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:15.671893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4bbcfacc-7940-4b1e-93ff-426cffb89bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:15.705769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcfacc-7940-4b1e-93ff-426cffb89bf5 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:15.771337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:15.822157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 819/1662] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:15.826348 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:15.827719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21609f8-e4a1-4e64-a506-b319138252c4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:15.827719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21609f8-e4a1-4e64-a506-b319138252c4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:15.827719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21609f8-e4a1-4e64-a506-b319138252c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:15.842601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:15.843261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:15.843538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:15.843813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] PATH_INFO: `/v3/projects/92c67b8aba4c4a688e8d8c6f6c7caf24` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:15.850885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c67b8aba4c4a688e8d8c6f6c7caf24)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:15.854248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:15.894702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f21609f8-e4a1-4e64-a506-b319138252c4 None None] MFA Rules not processed for user `624738daa9db4aff885d71ea2d65fb26`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:15.896954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f21609f8-e4a1-4e64-a506-b319138252c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:15.983029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c67b8aba4c4a688e8d8c6f6c7caf24', 'request_id': 'req-f7449aad-1d1c-4d37-9200-9c584bea72bd'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:16.002583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c67b8aba4c4a688e8d8c6f6c7caf24', 'request_id': 'req-f7449aad-1d1c-4d37-9200-9c584bea72bd'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:16.013502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c67b8aba4c4a688e8d8c6f6c7caf24', 'request_id': 'req-f7449aad-1d1c-4d37-9200-9c584bea72bd'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:16.049929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] The token cache is being invalidate because project 92c67b8aba4c4a688e8d8c6f6c7caf24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:16.050556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:16.050955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:16.055696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 844/1663] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:15 2022] DELETE /identity/v3/projects/92c67b8aba4c4a688e8d8c6f6c7caf24 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:16.058334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:16.061364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:16.093171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 820/1664] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:16.096302 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21609f8-e4a1-4e64-a506-b319138252c4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:16.100043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f21609f8-e4a1-4e64-a506-b319138252c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:16.303972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21609f8-e4a1-4e64-a506-b319138252c4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:16.354685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7449aad-1d1c-4d37-9200-9c584bea72bd admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:16.385192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b00603-0239-42c6-9ebd-242d94a5b6cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:16.385902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b00603-0239-42c6-9ebd-242d94a5b6cd service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:16.386192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b00603-0239-42c6-9ebd-242d94a5b6cd service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:16.386455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b00603-0239-42c6-9ebd-242d94a5b6cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:16.392850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-89b00603-0239-42c6-9ebd-242d94a5b6cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:16.480519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371c030-4b2e-4a0f-9672-0d71c0ed1986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:16.481149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371c030-4b2e-4a0f-9672-0d71c0ed1986 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:16.481612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371c030-4b2e-4a0f-9672-0d71c0ed1986 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:16.482096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371c030-4b2e-4a0f-9672-0d71c0ed1986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:16.487688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5371c030-4b2e-4a0f-9672-0d71c0ed1986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:16.602333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b00603-0239-42c6-9ebd-242d94a5b6cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:16.604009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b00603-0239-42c6-9ebd-242d94a5b6cd service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:16.636815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 845/1665] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:16.637922 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b00603-0239-42c6-9ebd-242d94a5b6cd service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:16.640637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-89b00603-0239-42c6-9ebd-242d94a5b6cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:16.643062 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:16.710769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:16.721373 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371c030-4b2e-4a0f-9672-0d71c0ed1986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:16.722859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371c030-4b2e-4a0f-9672-0d71c0ed1986 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:16.757148 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 821/1666] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:16.760154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92aafc-ca3d-4a56-a34b-9ab857824990 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:16.760154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92aafc-ca3d-4a56-a34b-9ab857824990 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:16.760383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92aafc-ca3d-4a56-a34b-9ab857824990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:16.813433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6e92aafc-ca3d-4a56-a34b-9ab857824990 None None] MFA Rules not processed for user `624738daa9db4aff885d71ea2d65fb26`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:16.815202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6e92aafc-ca3d-4a56-a34b-9ab857824990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:16.842119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b00603-0239-42c6-9ebd-242d94a5b6cd service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:16.860116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7f214-8ab4-4b46-b508-6f4f92026682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:16.860811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7f214-8ab4-4b46-b508-6f4f92026682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:16.861100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7f214-8ab4-4b46-b508-6f4f92026682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:16.861373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7f214-8ab4-4b46-b508-6f4f92026682 admin admin] PATH_INFO: `/v3/users/884fb021387e484fb2033e764565cdc8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:16.863826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7f214-8ab4-4b46-b508-6f4f92026682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=884fb021387e484fb2033e764565cdc8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:16.867437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7f214-8ab4-4b46-b508-6f4f92026682 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:17.000202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 822/1667] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:17.002579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92aafc-ca3d-4a56-a34b-9ab857824990 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:17.008084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6e92aafc-ca3d-4a56-a34b-9ab857824990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:17.056936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18d7f214-8ab4-4b46-b508-6f4f92026682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '884fb021387e484fb2033e764565cdc8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:17.079819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18d7f214-8ab4-4b46-b508-6f4f92026682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '884fb021387e484fb2033e764565cdc8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:17.101716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18d7f214-8ab4-4b46-b508-6f4f92026682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '884fb021387e484fb2033e764565cdc8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:17.133012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18d7f214-8ab4-4b46-b508-6f4f92026682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '884fb021387e484fb2033e764565cdc8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:17.180070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18d7f214-8ab4-4b46-b508-6f4f92026682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '884fb021387e484fb2033e764565cdc8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:17.197158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 846/1668] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:16 2022] DELETE /identity/v3/users/884fb021387e484fb2033e764565cdc8 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:17.197158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96ba54-7538-4090-a17f-21673f00f1d9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:17.197158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96ba54-7538-4090-a17f-21673f00f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:17.197158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96ba54-7538-4090-a17f-21673f00f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:17.226879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92aafc-ca3d-4a56-a34b-9ab857824990 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:17.273435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4d96ba54-7538-4090-a17f-21673f00f1d9 None None] MFA Rules not processed for user `ccbdadc310fc4bf8a6679bcb39b34ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:17.274732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4d96ba54-7538-4090-a17f-21673f00f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:17.362265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468345-1646-4833-a91b-81e965bacf8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:17.365157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468345-1646-4833-a91b-81e965bacf8e service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:17.365836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468345-1646-4833-a91b-81e965bacf8e service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:17.367156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468345-1646-4833-a91b-81e965bacf8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:17.378291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-39468345-1646-4833-a91b-81e965bacf8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:17.417637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39468345-1646-4833-a91b-81e965bacf8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:17.423505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39468345-1646-4833-a91b-81e965bacf8e service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:17.456095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 823/1669] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:17.460218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468345-1646-4833-a91b-81e965bacf8e service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:17.463019 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-39468345-1646-4833-a91b-81e965bacf8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:17.475518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 847/1670] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:17.480096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838d3f1-68da-42cb-82f7-59b00e3056e4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:17.483505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838d3f1-68da-42cb-82f7-59b00e3056e4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:17.485842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838d3f1-68da-42cb-82f7-59b00e3056e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:17.501833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:17.580471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a838d3f1-68da-42cb-82f7-59b00e3056e4 None None] MFA Rules not processed for user `f7aa40f3966d4c97ab6a1a764722969d`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:17.583948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a838d3f1-68da-42cb-82f7-59b00e3056e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:17.644853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:17.655883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468345-1646-4833-a91b-81e965bacf8e service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:17.677359 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b73880-afe2-4493-9f6f-0072e6649c1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:17.678080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b73880-afe2-4493-9f6f-0072e6649c1d service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:17.678795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b73880-afe2-4493-9f6f-0072e6649c1d service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:17.681015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b73880-afe2-4493-9f6f-0072e6649c1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:17.685326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-51b73880-afe2-4493-9f6f-0072e6649c1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:17.770143 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 848/1671] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:17.771623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838d3f1-68da-42cb-82f7-59b00e3056e4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:17.780307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a838d3f1-68da-42cb-82f7-59b00e3056e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:17.819560 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838d3f1-68da-42cb-82f7-59b00e3056e4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:17.837543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a051657-c93a-4dfe-b9e7-e69740d26638 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:17.842787 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a051657-c93a-4dfe-b9e7-e69740d26638 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:17.844201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a051657-c93a-4dfe-b9e7-e69740d26638 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:17.844533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a051657-c93a-4dfe-b9e7-e69740d26638 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:17.856326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b73880-afe2-4493-9f6f-0072e6649c1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:17.862204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b73880-afe2-4493-9f6f-0072e6649c1d service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:17.881539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a051657-c93a-4dfe-b9e7-e69740d26638 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:17.883090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a051657-c93a-4dfe-b9e7-e69740d26638 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:17.888902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 824/1672] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:17.910884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 849/1673] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:17.930976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b73880-afe2-4493-9f6f-0072e6649c1d service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:17.934345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-51b73880-afe2-4493-9f6f-0072e6649c1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:18.010508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a051657-c93a-4dfe-b9e7-e69740d26638 service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:18.014511 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8a051657-c93a-4dfe-b9e7-e69740d26638 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:18.210681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a051657-c93a-4dfe-b9e7-e69740d26638 service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:18.221944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:18.222901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:18.225478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:18.225933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] PATH_INFO: `/v3/projects/1890a51f09c146349299ed243a284480` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:18.231077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1890a51f09c146349299ed243a284480)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:18.235765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:18.257117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b73880-afe2-4493-9f6f-0072e6649c1d service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:18.270617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:18.271730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:18.272051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:18.272322 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] PATH_INFO: `/v3/projects/d9f37085e1ca47a88dc01edd217a9805` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:18.274357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f37085e1ca47a88dc01edd217a9805)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:18.275611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: WARNING py.warnings [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 19:29:18.275611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: warnings.warn(msg) Sep 30 19:29:18.275611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: Sep 30 19:29:18.277984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:18.354641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1890a51f09c146349299ed243a284480', 'request_id': 'req-e844d833-1624-408d-93d6-8d125d001b55'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:18.391854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1890a51f09c146349299ed243a284480', 'request_id': 'req-e844d833-1624-408d-93d6-8d125d001b55'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:18.429893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1890a51f09c146349299ed243a284480', 'request_id': 'req-e844d833-1624-408d-93d6-8d125d001b55'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:18.454187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] The token cache is being invalidate because project 1890a51f09c146349299ed243a284480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:18.454788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:18.455990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:18.462107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 850/1674] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:18 2022] DELETE /identity/v3/projects/1890a51f09c146349299ed243a284480 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:29:18.463293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:18.466430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:18.489781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f37085e1ca47a88dc01edd217a9805', 'request_id': 'req-d13c937f-d50c-4502-8fee-67c919329af7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:18.509166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:18.511720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f37085e1ca47a88dc01edd217a9805', 'request_id': 'req-d13c937f-d50c-4502-8fee-67c919329af7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:18.576686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f37085e1ca47a88dc01edd217a9805', 'request_id': 'req-d13c937f-d50c-4502-8fee-67c919329af7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:18.600214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:18.636018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] The token cache is being invalidate because project d9f37085e1ca47a88dc01edd217a9805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:18.636018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:18.638697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d13c937f-d50c-4502-8fee-67c919329af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:18.645516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 825/1675] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:17 2022] DELETE /identity/v3/projects/d9f37085e1ca47a88dc01edd217a9805 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:18.648424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1584fa-b980-4c47-98d2-3bda3da16613 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:18.648424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1584fa-b980-4c47-98d2-3bda3da16613 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:18.648424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1584fa-b980-4c47-98d2-3bda3da16613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:18.670749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844d833-1624-408d-93d6-8d125d001b55 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:18.702805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dc1584fa-b980-4c47-98d2-3bda3da16613 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:18.712208 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dc1584fa-b980-4c47-98d2-3bda3da16613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:18.894299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df728c05-171f-4084-829c-336a86e49de1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:18.896844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df728c05-171f-4084-829c-336a86e49de1 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:18.898528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df728c05-171f-4084-829c-336a86e49de1 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:18.898851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df728c05-171f-4084-829c-336a86e49de1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:18.902784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-df728c05-171f-4084-829c-336a86e49de1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:18.925735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 826/1676] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:18.928991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31181b69-bada-4377-b5c2-c9dc917762e3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:18.932492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31181b69-bada-4377-b5c2-c9dc917762e3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:18.932916 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31181b69-bada-4377-b5c2-c9dc917762e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:18.985211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-31181b69-bada-4377-b5c2-c9dc917762e3 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:18.986601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-31181b69-bada-4377-b5c2-c9dc917762e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:19.011754 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 827/1677] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:19.013201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65c3c5-9c89-43a8-9a0f-03c59d41fcc8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:19.013585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65c3c5-9c89-43a8-9a0f-03c59d41fcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:19.014399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65c3c5-9c89-43a8-9a0f-03c59d41fcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:19.062971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8b65c3c5-9c89-43a8-9a0f-03c59d41fcc8 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:19.064354 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8b65c3c5-9c89-43a8-9a0f-03c59d41fcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:19.069740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df728c05-171f-4084-829c-336a86e49de1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:19.071600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df728c05-171f-4084-829c-336a86e49de1 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:19.101590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 828/1678] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:19.101590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdb4c3-59a4-4985-9617-1896397e782e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:19.101590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdb4c3-59a4-4985-9617-1896397e782e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:19.101590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdb4c3-59a4-4985-9617-1896397e782e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:19.104860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 851/1679] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:19.105853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df728c05-171f-4084-829c-336a86e49de1 service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:19.146924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df728c05-171f-4084-829c-336a86e49de1 service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:19.168981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af023a-b953-4923-9a0d-31be99029b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:19.169612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af023a-b953-4923-9a0d-31be99029b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:19.169887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af023a-b953-4923-9a0d-31be99029b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:19.170961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-96fdb4c3-59a4-4985-9617-1896397e782e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:19.172410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-96fdb4c3-59a4-4985-9617-1896397e782e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:19.173080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af023a-b953-4923-9a0d-31be99029b70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:19.173808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af023a-b953-4923-9a0d-31be99029b70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:29:19.174118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af023a-b953-4923-9a0d-31be99029b70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:19.180054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af023a-b953-4923-9a0d-31be99029b70 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:19.211539 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 829/1680] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:19.214675 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdb4c3-59a4-4985-9617-1896397e782e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:19.223252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 852/1681] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:29:19.228550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af023a-b953-4923-9a0d-31be99029b70 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:19.248034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af023a-b953-4923-9a0d-31be99029b70 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:19.259285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdd883-75b6-4c01-9477-8fa59530b2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:19.260293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdd883-75b6-4c01-9477-8fa59530b2ff admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:19.260699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdd883-75b6-4c01-9477-8fa59530b2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:19.261067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdd883-75b6-4c01-9477-8fa59530b2ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:19.261809 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdd883-75b6-4c01-9477-8fa59530b2ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:19.263837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdd883-75b6-4c01-9477-8fa59530b2ff admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:19.287483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdb4c3-59a4-4985-9617-1896397e782e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:19.301473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9a194-54ec-4150-afe7-f89b360a5288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:19.302550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9a194-54ec-4150-afe7-f89b360a5288 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:19.302826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9a194-54ec-4150-afe7-f89b360a5288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:19.307432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9a194-54ec-4150-afe7-f89b360a5288 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:19.312547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9a194-54ec-4150-afe7-f89b360a5288 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:29:19.314035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9a194-54ec-4150-afe7-f89b360a5288 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:19.324086 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9a194-54ec-4150-afe7-f89b360a5288 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:19.331179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 853/1682] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:19 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:29:19.334848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdd883-75b6-4c01-9477-8fa59530b2ff admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:19.372824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 830/1683] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:29:19.374561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9a194-54ec-4150-afe7-f89b360a5288 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:19.410223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9a194-54ec-4150-afe7-f89b360a5288 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:19.428723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:19.457010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdd883-75b6-4c01-9477-8fa59530b2ff admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:19.462711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88046207-2caf-494e-8daa-58d617f613eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:19.463620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88046207-2caf-494e-8daa-58d617f613eb admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:19.464010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88046207-2caf-494e-8daa-58d617f613eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:19.464411 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88046207-2caf-494e-8daa-58d617f613eb admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:19.465208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88046207-2caf-494e-8daa-58d617f613eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:19.467701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88046207-2caf-494e-8daa-58d617f613eb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:19.579192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 854/1684] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:19 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:29:19.580964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88046207-2caf-494e-8daa-58d617f613eb admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:19.606806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d38d59-34f5-47b7-b46e-9f9f3645d4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:19.607427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d38d59-34f5-47b7-b46e-9f9f3645d4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:19.611777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d38d59-34f5-47b7-b46e-9f9f3645d4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:19.612616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d38d59-34f5-47b7-b46e-9f9f3645d4d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:19.613023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d38d59-34f5-47b7-b46e-9f9f3645d4d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:19.620612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88046207-2caf-494e-8daa-58d617f613eb admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:19.621399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d38d59-34f5-47b7-b46e-9f9f3645d4d3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:19.630686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495357b0-48dc-4f0d-9bba-ac25f89bf63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:19.638111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495357b0-48dc-4f0d-9bba-ac25f89bf63f admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:19.638636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495357b0-48dc-4f0d-9bba-ac25f89bf63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:19.639237 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495357b0-48dc-4f0d-9bba-ac25f89bf63f admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:19.640143 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495357b0-48dc-4f0d-9bba-ac25f89bf63f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:19.642423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495357b0-48dc-4f0d-9bba-ac25f89bf63f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:19.682408 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 831/1685] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:19 2022] POST /identity/v3/projects => generated 371 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:19.687918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d38d59-34f5-47b7-b46e-9f9f3645d4d3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:19.719805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 855/1686] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:19.720968 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495357b0-48dc-4f0d-9bba-ac25f89bf63f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:19.772030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495357b0-48dc-4f0d-9bba-ac25f89bf63f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:19.813244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:19.899806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d38d59-34f5-47b7-b46e-9f9f3645d4d3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:19.916217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57090512-1819-4035-9586-bd1a6518d231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:19.916883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57090512-1819-4035-9586-bd1a6518d231 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:19.917362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57090512-1819-4035-9586-bd1a6518d231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:19.918358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57090512-1819-4035-9586-bd1a6518d231 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:19.921703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57090512-1819-4035-9586-bd1a6518d231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:19.927673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57090512-1819-4035-9586-bd1a6518d231 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:20.039880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c240d5-1239-4716-a34f-59b7001684f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:20.040197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 832/1687] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:19 2022] POST /identity/v3/users => generated 286 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:20.040544 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c240d5-1239-4716-a34f-59b7001684f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:20.040823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c240d5-1239-4716-a34f-59b7001684f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:20.041206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c240d5-1239-4716-a34f-59b7001684f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:20.041304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57090512-1819-4035-9586-bd1a6518d231 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:20.042168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c240d5-1239-4716-a34f-59b7001684f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:20.043337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c240d5-1239-4716-a34f-59b7001684f8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:20.111078 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57090512-1819-4035-9586-bd1a6518d231 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:20.114608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 856/1688] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:20.115671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c240d5-1239-4716-a34f-59b7001684f8 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:20.134960 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c4a4b-527c-4201-8f33-2bf0f04e557f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:20.135557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c4a4b-527c-4201-8f33-2bf0f04e557f admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:20.135947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c4a4b-527c-4201-8f33-2bf0f04e557f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:20.136137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c4a4b-527c-4201-8f33-2bf0f04e557f admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:20.140093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c240d5-1239-4716-a34f-59b7001684f8 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:20.146650 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c4a4b-527c-4201-8f33-2bf0f04e557f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:20.152398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a46dbbc-51ec-4e17-937d-136eb45573e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:20.152701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46dbbc-51ec-4e17-937d-136eb45573e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:20.152981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46dbbc-51ec-4e17-937d-136eb45573e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:20.153257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46dbbc-51ec-4e17-937d-136eb45573e4 admin admin] PATH_INFO: `/v3/projects/439d6a6b96a7449a83c483989833caf0/users/a60e9b98677f473eb58da5fdb27b6ec2/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:20.156942 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c4a4b-527c-4201-8f33-2bf0f04e557f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:20.201090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a46dbbc-51ec-4e17-937d-136eb45573e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=439d6a6b96a7449a83c483989833caf0, user_id=a60e9b98677f473eb58da5fdb27b6ec2, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:20.203258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a46dbbc-51ec-4e17-937d-136eb45573e4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:20.212935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 833/1689] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:20.215612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c4a4b-527c-4201-8f33-2bf0f04e557f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:20.252227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 857/1690] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:20 2022] PUT /identity/v3/projects/439d6a6b96a7449a83c483989833caf0/users/a60e9b98677f473eb58da5fdb27b6ec2/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:20.264407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eda2e-2aaa-45d5-b87b-51df68ed3412 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:20.264916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eda2e-2aaa-45d5-b87b-51df68ed3412 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:20.265385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eda2e-2aaa-45d5-b87b-51df68ed3412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:20.275467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c4a4b-527c-4201-8f33-2bf0f04e557f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:20.381500 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7b6eda2e-2aaa-45d5-b87b-51df68ed3412 None None] MFA Rules not processed for user `a60e9b98677f473eb58da5fdb27b6ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:20.382748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7b6eda2e-2aaa-45d5-b87b-51df68ed3412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:20.482297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d0bb1-19f5-4d11-a413-fdcb067de0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:20.484806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d0bb1-19f5-4d11-a413-fdcb067de0af admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:20.486481 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d0bb1-19f5-4d11-a413-fdcb067de0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:20.488101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d0bb1-19f5-4d11-a413-fdcb067de0af admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:20.489575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d0bb1-19f5-4d11-a413-fdcb067de0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:20.495361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d0bb1-19f5-4d11-a413-fdcb067de0af admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:20.536435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 834/1691] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:20.539643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4981c-9158-43f3-864b-c718d58a1e49 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:20.541412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4981c-9158-43f3-864b-c718d58a1e49 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:20.543375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4981c-9158-43f3-864b-c718d58a1e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:20.596489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-05f4981c-9158-43f3-864b-c718d58a1e49 None None] MFA Rules not processed for user `624738daa9db4aff885d71ea2d65fb26`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:20.601349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-05f4981c-9158-43f3-864b-c718d58a1e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:20.682374 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 858/1692] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:29:20.683015 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6eda2e-2aaa-45d5-b87b-51df68ed3412 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:20.742467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6eda2e-2aaa-45d5-b87b-51df68ed3412 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:20.761524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381be71-503b-4c4c-b474-6d2e88bb92dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:20.763280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381be71-503b-4c4c-b474-6d2e88bb92dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:20.764330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381be71-503b-4c4c-b474-6d2e88bb92dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:20.765897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381be71-503b-4c4c-b474-6d2e88bb92dc admin admin] PATH_INFO: `/v3/projects/2c308897e7854e3a8b3a7d7a65065b0b/users/8b1bf69b827443e79921128845a70566/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:20.817509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381be71-503b-4c4c-b474-6d2e88bb92dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c308897e7854e3a8b3a7d7a65065b0b, user_id=8b1bf69b827443e79921128845a70566, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:20.820519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381be71-503b-4c4c-b474-6d2e88bb92dc admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:20.821535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 835/1693] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:20.824749 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4981c-9158-43f3-864b-c718d58a1e49 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:20.828883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:20.852000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 859/1694] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:20 2022] PUT /identity/v3/projects/2c308897e7854e3a8b3a7d7a65065b0b/users/8b1bf69b827443e79921128845a70566/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:20.867735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557a0d6-83f2-4806-b4e8-fe7e2314dcdb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:20.868398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557a0d6-83f2-4806-b4e8-fe7e2314dcdb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:20.868842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557a0d6-83f2-4806-b4e8-fe7e2314dcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:20.881772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4981c-9158-43f3-864b-c718d58a1e49 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:20.951246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4557a0d6-83f2-4806-b4e8-fe7e2314dcdb None None] MFA Rules not processed for user `8b1bf69b827443e79921128845a70566`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:20.955745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4557a0d6-83f2-4806-b4e8-fe7e2314dcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:21.127607 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:21.152663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2221b-0859-4a41-83f5-d9f0d928a6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:21.153427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2221b-0859-4a41-83f5-d9f0d928a6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:21.153717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2221b-0859-4a41-83f5-d9f0d928a6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:21.153993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2221b-0859-4a41-83f5-d9f0d928a6d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:21.154693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2221b-0859-4a41-83f5-d9f0d928a6d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:21.156676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2221b-0859-4a41-83f5-d9f0d928a6d2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:21.248420 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 836/1695] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:20 2022] POST /identity/v3/projects => generated 373 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:21.251215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2221b-0859-4a41-83f5-d9f0d928a6d2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:21.303569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 860/1696] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:29:21.307112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557a0d6-83f2-4806-b4e8-fe7e2314dcdb None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:21.357649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557a0d6-83f2-4806-b4e8-fe7e2314dcdb None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:21.383009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab36765-2020-495c-8910-4e3907aea121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:21.383734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab36765-2020-495c-8910-4e3907aea121 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:21.383970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab36765-2020-495c-8910-4e3907aea121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:21.384243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab36765-2020-495c-8910-4e3907aea121 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:21.385920 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab36765-2020-495c-8910-4e3907aea121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:21.394098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab36765-2020-495c-8910-4e3907aea121 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:21.400011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2221b-0859-4a41-83f5-d9f0d928a6d2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:21.427637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7bbb7-b3fa-4182-baab-585af5d7a897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:21.428559 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7bbb7-b3fa-4182-baab-585af5d7a897 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:21.428846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7bbb7-b3fa-4182-baab-585af5d7a897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:21.429117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7bbb7-b3fa-4182-baab-585af5d7a897 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:21.429759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7bbb7-b3fa-4182-baab-585af5d7a897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:21.432057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7bbb7-b3fa-4182-baab-585af5d7a897 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:21.470093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 861/1697] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:21 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:21.474230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab36765-2020-495c-8910-4e3907aea121 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:21.499749 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 837/1698] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:21 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:29:21.502519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7bbb7-b3fa-4182-baab-585af5d7a897 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:21.531173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7bbb7-b3fa-4182-baab-585af5d7a897 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:21.671881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a00d84-6ff4-49fe-8897-ddb8f68f475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:21.672551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a00d84-6ff4-49fe-8897-ddb8f68f475c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:21.672833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a00d84-6ff4-49fe-8897-ddb8f68f475c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:21.673107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a00d84-6ff4-49fe-8897-ddb8f68f475c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:21.673759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a00d84-6ff4-49fe-8897-ddb8f68f475c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:21.681986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a00d84-6ff4-49fe-8897-ddb8f68f475c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:21.729365 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 838/1699] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:21.729540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a00d84-6ff4-49fe-8897-ddb8f68f475c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:21.742113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab36765-2020-495c-8910-4e3907aea121 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:21.760578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95666c-6cb3-4021-b649-259823693a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:21.761216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95666c-6cb3-4021-b649-259823693a6b admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:21.761644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95666c-6cb3-4021-b649-259823693a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:21.761922 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95666c-6cb3-4021-b649-259823693a6b admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:21.762569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95666c-6cb3-4021-b649-259823693a6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:21.766472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a00d84-6ff4-49fe-8897-ddb8f68f475c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:21.767420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95666c-6cb3-4021-b649-259823693a6b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:21.781446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f015f2-63f6-4905-bc11-207c714363d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:21.782562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f015f2-63f6-4905-bc11-207c714363d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:21.782907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f015f2-63f6-4905-bc11-207c714363d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:21.783253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f015f2-63f6-4905-bc11-207c714363d7 admin admin] PATH_INFO: `/v3/projects/83326596f1ad4c05ae63f7b7954967fa/users/111cba50b908499989db6be47b1ff555/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:21.819080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f015f2-63f6-4905-bc11-207c714363d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83326596f1ad4c05ae63f7b7954967fa, user_id=111cba50b908499989db6be47b1ff555, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:21.822129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f015f2-63f6-4905-bc11-207c714363d7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:21.823989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 862/1700] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:21 2022] POST /identity/v3/users => generated 292 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:21.828922 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95666c-6cb3-4021-b649-259823693a6b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:21.850144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 839/1701] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:21 2022] PUT /identity/v3/projects/83326596f1ad4c05ae63f7b7954967fa/users/111cba50b908499989db6be47b1ff555/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:21.852093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95666c-6cb3-4021-b649-259823693a6b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:21.858822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f015f2-63f6-4905-bc11-207c714363d7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:21.975297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:22.014817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f015f2-63f6-4905-bc11-207c714363d7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:22.029298 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa48a74-4e4a-439d-abb4-dbd6845b9cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:22.029930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa48a74-4e4a-439d-abb4-dbd6845b9cce admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:22.030210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa48a74-4e4a-439d-abb4-dbd6845b9cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:22.030578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa48a74-4e4a-439d-abb4-dbd6845b9cce admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:22.031809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa48a74-4e4a-439d-abb4-dbd6845b9cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:22.032903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa48a74-4e4a-439d-abb4-dbd6845b9cce admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:22.036434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:22.065353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 840/1702] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:22.065353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c861-68ea-48fb-a69e-e0ce8f11a1c1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:22.065353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c861-68ea-48fb-a69e-e0ce8f11a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:22.065353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c861-68ea-48fb-a69e-e0ce8f11a1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:22.068084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a4d72-2e56-4d8b-a8ec-0d5516275f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:22.068945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a4d72-2e56-4d8b-a8ec-0d5516275f95 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:22.069894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a4d72-2e56-4d8b-a8ec-0d5516275f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:22.070840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a4d72-2e56-4d8b-a8ec-0d5516275f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:22.072883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a4d72-2e56-4d8b-a8ec-0d5516275f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:22.079649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a4d72-2e56-4d8b-a8ec-0d5516275f95 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:22.109489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 863/1703] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:29:22.110635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a4d72-2e56-4d8b-a8ec-0d5516275f95 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:22.142611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a4d72-2e56-4d8b-a8ec-0d5516275f95 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:22.163439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79379a79-4adf-49ca-9ffe-ef0b07b66a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:22.163949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79379a79-4adf-49ca-9ffe-ef0b07b66a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:22.164229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79379a79-4adf-49ca-9ffe-ef0b07b66a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:22.164530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79379a79-4adf-49ca-9ffe-ef0b07b66a23 admin admin] PATH_INFO: `/v3/projects/83326596f1ad4c05ae63f7b7954967fa/users/111cba50b908499989db6be47b1ff555/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:22.171151 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79379a79-4adf-49ca-9ffe-ef0b07b66a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83326596f1ad4c05ae63f7b7954967fa, user_id=111cba50b908499989db6be47b1ff555, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:22.176475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79379a79-4adf-49ca-9ffe-ef0b07b66a23 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:22.180665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-33e2c861-68ea-48fb-a69e-e0ce8f11a1c1 None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:22.182014 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-33e2c861-68ea-48fb-a69e-e0ce8f11a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:22.210063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 864/1704] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:22 2022] PUT /identity/v3/projects/83326596f1ad4c05ae63f7b7954967fa/users/111cba50b908499989db6be47b1ff555/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:22.211650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79379a79-4adf-49ca-9ffe-ef0b07b66a23 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:22.335024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 841/1705] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:29:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:22.338785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180394b7-f187-4a10-95f6-c820a94a6c03 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:22.339170 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180394b7-f187-4a10-95f6-c820a94a6c03 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:22.339455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180394b7-f187-4a10-95f6-c820a94a6c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:22.422890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79379a79-4adf-49ca-9ffe-ef0b07b66a23 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:22.429490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6bb73-95df-42ee-9c5d-425afd3b6994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:22.430189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6bb73-95df-42ee-9c5d-425afd3b6994 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:22.430677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6bb73-95df-42ee-9c5d-425afd3b6994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:22.430963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6bb73-95df-42ee-9c5d-425afd3b6994 admin admin] PATH_INFO: `/v3/projects/2c9dd2b1d1df4060b9f1977880fa491b/users/d5dbd31355cf44d5a88f4a8b6d3fe074/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:22.431811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-180394b7-f187-4a10-95f6-c820a94a6c03 None None] MFA Rules not processed for user `111cba50b908499989db6be47b1ff555`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:22.433068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-180394b7-f187-4a10-95f6-c820a94a6c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:22.451271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6bb73-95df-42ee-9c5d-425afd3b6994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c9dd2b1d1df4060b9f1977880fa491b, user_id=d5dbd31355cf44d5a88f4a8b6d3fe074, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:22.454867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6bb73-95df-42ee-9c5d-425afd3b6994 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:22.478658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 865/1706] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:22 2022] PUT /identity/v3/projects/2c9dd2b1d1df4060b9f1977880fa491b/users/d5dbd31355cf44d5a88f4a8b6d3fe074/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:22.483534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6bb73-95df-42ee-9c5d-425afd3b6994 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:22.528035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 842/1707] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:22.541457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180394b7-f187-4a10-95f6-c820a94a6c03 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:22.543188 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-180394b7-f187-4a10-95f6-c820a94a6c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:22.617325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6bb73-95df-42ee-9c5d-425afd3b6994 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:22.622756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc126543-62ca-49ec-9d16-96387cc291dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:22.623437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc126543-62ca-49ec-9d16-96387cc291dd admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:22.623938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc126543-62ca-49ec-9d16-96387cc291dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:22.624519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc126543-62ca-49ec-9d16-96387cc291dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:22.625584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:22.627357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc126543-62ca-49ec-9d16-96387cc291dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:22.631338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc126543-62ca-49ec-9d16-96387cc291dd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:22.645888 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 866/1708] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:22.646967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc126543-62ca-49ec-9d16-96387cc291dd admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:22.651145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cc126543-62ca-49ec-9d16-96387cc291dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:22.652101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180394b7-f187-4a10-95f6-c820a94a6c03 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:22.674176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc126543-62ca-49ec-9d16-96387cc291dd admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:22.687956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:22.689821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:22.690686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:22.690971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] PATH_INFO: `/v3/users/aacda3a1a55046e882e56727033b0d48` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:22.696605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aacda3a1a55046e882e56727033b0d48)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:22.698790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:22.798323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aacda3a1a55046e882e56727033b0d48'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:22.803345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:22.808501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b1d57-b507-4d21-8646-9dcce29a0457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:22.809707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b1d57-b507-4d21-8646-9dcce29a0457 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:22.810099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b1d57-b507-4d21-8646-9dcce29a0457 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:22.810467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b1d57-b507-4d21-8646-9dcce29a0457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:22.824863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aacda3a1a55046e882e56727033b0d48'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:22.827320 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aacda3a1a55046e882e56727033b0d48'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:22.851360 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aacda3a1a55046e882e56727033b0d48'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:22.897113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aacda3a1a55046e882e56727033b0d48'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:22.913193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 867/1709] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:22 2022] DELETE /identity/v3/users/aacda3a1a55046e882e56727033b0d48 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:22.915230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:22.918779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:22.999419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b1d57-b507-4d21-8646-9dcce29a0457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:23.001088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b1d57-b507-4d21-8646-9dcce29a0457 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:23.023095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 843/1710] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:29:23.023948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b1d57-b507-4d21-8646-9dcce29a0457 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:23.025559 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5d7b1d57-b507-4d21-8646-9dcce29a0457 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:23.100580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd4e87-18c4-4e90-84ef-35628fec80b9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:23.116483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae4bdd-3b36-4b60-ada7-34a5e8056cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:23.117184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae4bdd-3b36-4b60-ada7-34a5e8056cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:23.117461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae4bdd-3b36-4b60-ada7-34a5e8056cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:23.117724 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae4bdd-3b36-4b60-ada7-34a5e8056cd2 admin admin] PATH_INFO: `/v3/projects/2c9dd2b1d1df4060b9f1977880fa491b/users/d5dbd31355cf44d5a88f4a8b6d3fe074/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:23.138937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae4bdd-3b36-4b60-ada7-34a5e8056cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c9dd2b1d1df4060b9f1977880fa491b, user_id=d5dbd31355cf44d5a88f4a8b6d3fe074, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:23.147088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae4bdd-3b36-4b60-ada7-34a5e8056cd2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:23.160095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b1d57-b507-4d21-8646-9dcce29a0457 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:23.196658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 868/1711] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:22 2022] PUT /identity/v3/projects/2c9dd2b1d1df4060b9f1977880fa491b/users/d5dbd31355cf44d5a88f4a8b6d3fe074/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:23.198859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad767b3b-71c8-4563-aefd-6d424e815ba2 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:23.201007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad767b3b-71c8-4563-aefd-6d424e815ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:23.201974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad767b3b-71c8-4563-aefd-6d424e815ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:23.246462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:23.286762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ad767b3b-71c8-4563-aefd-6d424e815ba2 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:23.296996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ad767b3b-71c8-4563-aefd-6d424e815ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:23.461251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8baeb4c-6e64-4a24-8c38-b567584fff07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:23.461883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baeb4c-6e64-4a24-8c38-b567584fff07 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:23.462163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baeb4c-6e64-4a24-8c38-b567584fff07 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:23.462449 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baeb4c-6e64-4a24-8c38-b567584fff07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:23.473036 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f8baeb4c-6e64-4a24-8c38-b567584fff07 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:23.546845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8baeb4c-6e64-4a24-8c38-b567584fff07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:23.549380 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8baeb4c-6e64-4a24-8c38-b567584fff07 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:23.562644 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 844/1712] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:29:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:29:23.564938 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92c9d0-e5fb-4a1b-a7c0-b63e381fea79 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:23.565327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92c9d0-e5fb-4a1b-a7c0-b63e381fea79 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:23.565714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92c9d0-e5fb-4a1b-a7c0-b63e381fea79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:23.583554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 869/1713] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:23.585403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd28cf1-380f-4a58-9105-49fcb769f3d7 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:23.586346 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd28cf1-380f-4a58-9105-49fcb769f3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:23.587952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd28cf1-380f-4a58-9105-49fcb769f3d7 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:23.594760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 870/1714] 10.222.0.17 () {54 vars in 912 bytes} [Fri Sep 30 19:29:23 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:29:23.597915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371a4db-c950-4724-bbcf-44e49a0456af None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:23.601379 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371a4db-c950-4724-bbcf-44e49a0456af None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:23.601824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371a4db-c950-4724-bbcf-44e49a0456af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:23.663597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7e92c9d0-e5fb-4a1b-a7c0-b63e381fea79 None None] MFA Rules not processed for user `d5dbd31355cf44d5a88f4a8b6d3fe074`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:23.665571 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7e92c9d0-e5fb-4a1b-a7c0-b63e381fea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:23.667421 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9371a4db-c950-4724-bbcf-44e49a0456af None None] MFA Rules not processed for user `ccbdadc310fc4bf8a6679bcb39b34ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:23.670200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9371a4db-c950-4724-bbcf-44e49a0456af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:23.951379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 845/1715] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:23.957588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc993cdd-4aa3-4235-8e2e-0fc3601463ff None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:23.957588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc993cdd-4aa3-4235-8e2e-0fc3601463ff None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:23.960497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc993cdd-4aa3-4235-8e2e-0fc3601463ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:23.980133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 871/1716] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:23.981211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371a4db-c950-4724-bbcf-44e49a0456af None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:24.018066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9371a4db-c950-4724-bbcf-44e49a0456af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:24.030162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cc993cdd-4aa3-4235-8e2e-0fc3601463ff None None] MFA Rules not processed for user `18ea49d1f6c040f18511535b7e51a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:24.033465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cc993cdd-4aa3-4235-8e2e-0fc3601463ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:24.047301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:24.132845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371a4db-c950-4724-bbcf-44e49a0456af None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:24.137488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b52e9-c85c-472e-a40c-9988d3aa0513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:24.138963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b52e9-c85c-472e-a40c-9988d3aa0513 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:24.139419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b52e9-c85c-472e-a40c-9988d3aa0513 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:24.139835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b52e9-c85c-472e-a40c-9988d3aa0513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:24.154667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b52e9-c85c-472e-a40c-9988d3aa0513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:24.157173 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b52e9-c85c-472e-a40c-9988d3aa0513 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:24.170925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 872/1717] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:24.172204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b52e9-c85c-472e-a40c-9988d3aa0513 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:24.172530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 846/1718] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:24.173690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc993cdd-4aa3-4235-8e2e-0fc3601463ff None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:24.174233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-664b52e9-c85c-472e-a40c-9988d3aa0513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:24.177741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cc993cdd-4aa3-4235-8e2e-0fc3601463ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:24.195936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:24.209082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc993cdd-4aa3-4235-8e2e-0fc3601463ff None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:24.222548 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:24.224104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:24.227288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:24.228054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] PATH_INFO: `/v3/users/a60e9b98677f473eb58da5fdb27b6ec2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:24.229769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a60e9b98677f473eb58da5fdb27b6ec2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:24.236681 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:24.324865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b52e9-c85c-472e-a40c-9988d3aa0513 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:24.337349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beaa70d-5621-4f7a-8b7b-71770828c70b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:24.339273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beaa70d-5621-4f7a-8b7b-71770828c70b service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:24.342357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beaa70d-5621-4f7a-8b7b-71770828c70b service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:24.343307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beaa70d-5621-4f7a-8b7b-71770828c70b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:24.368991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a60e9b98677f473eb58da5fdb27b6ec2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:24.383845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beaa70d-5621-4f7a-8b7b-71770828c70b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:24.386113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beaa70d-5621-4f7a-8b7b-71770828c70b service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:24.400074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a60e9b98677f473eb58da5fdb27b6ec2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:24.403378 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a60e9b98677f473eb58da5fdb27b6ec2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:24.429629 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a60e9b98677f473eb58da5fdb27b6ec2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:24.441881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 873/1719] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:24.481369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a60e9b98677f473eb58da5fdb27b6ec2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:24.503191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 847/1720] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:24 2022] DELETE /identity/v3/users/a60e9b98677f473eb58da5fdb27b6ec2 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:24.505756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591dd68-9415-443c-8b93-c56022c8d6a1 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:24.506140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591dd68-9415-443c-8b93-c56022c8d6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:24.506419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591dd68-9415-443c-8b93-c56022c8d6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:24.539851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:24.542057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:24.561145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0591dd68-9415-443c-8b93-c56022c8d6a1 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:24.569395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0591dd68-9415-443c-8b93-c56022c8d6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:24.753438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d881e-6dc2-49b1-8fcf-11f22c7c1712 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:24.763537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10043fe5-0a10-43c0-aad0-ea0bc611e69f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:24.764174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10043fe5-0a10-43c0-aad0-ea0bc611e69f service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:24.764477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10043fe5-0a10-43c0-aad0-ea0bc611e69f service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:24.764751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10043fe5-0a10-43c0-aad0-ea0bc611e69f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:24.777160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-10043fe5-0a10-43c0-aad0-ea0bc611e69f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:24.797016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 874/1721] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:24.798139 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591dd68-9415-443c-8b93-c56022c8d6a1 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:24.801907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0591dd68-9415-443c-8b93-c56022c8d6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:24.811261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10043fe5-0a10-43c0-aad0-ea0bc611e69f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:24.812802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10043fe5-0a10-43c0-aad0-ea0bc611e69f service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:24.840581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 848/1722] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:24.847053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10043fe5-0a10-43c0-aad0-ea0bc611e69f service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:24.860815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10043fe5-0a10-43c0-aad0-ea0bc611e69f service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:24.882679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656f1eb-1d27-407a-93de-88d343eee43b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:24.883357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656f1eb-1d27-407a-93de-88d343eee43b service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:24.883656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656f1eb-1d27-407a-93de-88d343eee43b service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:24.883928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656f1eb-1d27-407a-93de-88d343eee43b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:24.902034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656f1eb-1d27-407a-93de-88d343eee43b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:24.903635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656f1eb-1d27-407a-93de-88d343eee43b service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:24.945494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 849/1723] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:29:24.980738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591dd68-9415-443c-8b93-c56022c8d6a1 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:25.001223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b387f5c-f776-435a-b77f-b63a6d6db9d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:25.003675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b387f5c-f776-435a-b77f-b63a6d6db9d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:25.004413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b387f5c-f776-435a-b77f-b63a6d6db9d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:25.004413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b387f5c-f776-435a-b77f-b63a6d6db9d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:25.016421 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3b387f5c-f776-435a-b77f-b63a6d6db9d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:25.026023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:25.158549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b387f5c-f776-435a-b77f-b63a6d6db9d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:25.160467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b387f5c-f776-435a-b77f-b63a6d6db9d8 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:25.172058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 875/1724] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:25.813381 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555cab1-e19a-4ad6-86f6-e310d6eea91b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:25.814001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555cab1-e19a-4ad6-86f6-e310d6eea91b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:25.814001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555cab1-e19a-4ad6-86f6-e310d6eea91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:25.849054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b555cab1-e19a-4ad6-86f6-e310d6eea91b None None] MFA Rules not processed for user `a397314cdd52409c920071ae009516af`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:25.850452 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b555cab1-e19a-4ad6-86f6-e310d6eea91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:25.940849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 850/1725] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:27.407826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50cd13-03ea-439a-a7b7-c3eddbf91fdf None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:27.407826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50cd13-03ea-439a-a7b7-c3eddbf91fdf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:27.407826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50cd13-03ea-439a-a7b7-c3eddbf91fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:27.494771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ed50cd13-03ea-439a-a7b7-c3eddbf91fdf None None] MFA Rules not processed for user `18ea49d1f6c040f18511535b7e51a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:27.499180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ed50cd13-03ea-439a-a7b7-c3eddbf91fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:27.533021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 876/1726] 10.222.0.36 () {58 vars in 1015 bytes} [Fri Sep 30 19:29:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:27.581051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555cab1-e19a-4ad6-86f6-e310d6eea91b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:27.583266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b555cab1-e19a-4ad6-86f6-e310d6eea91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:27.615158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555cab1-e19a-4ad6-86f6-e310d6eea91b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:27.631363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:27.632298 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:27.635856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:27.636445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] PATH_INFO: `/v3/projects/2f1c4a61fe0e454c805c6e0ed507416f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:27.640288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f1c4a61fe0e454c805c6e0ed507416f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:27.642372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:27.652467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:27.671932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50cd13-03ea-439a-a7b7-c3eddbf91fdf None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:27.673649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ed50cd13-03ea-439a-a7b7-c3eddbf91fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:27.691524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50cd13-03ea-439a-a7b7-c3eddbf91fdf None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:27.713105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382ad3f-827a-48c7-8dbc-99e49b31d373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:27.727916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382ad3f-827a-48c7-8dbc-99e49b31d373 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:27.736449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382ad3f-827a-48c7-8dbc-99e49b31d373 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:27.739289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382ad3f-827a-48c7-8dbc-99e49b31d373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:27.794852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382ad3f-827a-48c7-8dbc-99e49b31d373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:27.813791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382ad3f-827a-48c7-8dbc-99e49b31d373 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:27.873496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 877/1727] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:27.934432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f1c4a61fe0e454c805c6e0ed507416f', 'request_id': 'req-40d21c72-34ac-4bab-aaa9-4db05493a2cf'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:27.948183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f1c4a61fe0e454c805c6e0ed507416f', 'request_id': 'req-40d21c72-34ac-4bab-aaa9-4db05493a2cf'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:27.985219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f1c4a61fe0e454c805c6e0ed507416f', 'request_id': 'req-40d21c72-34ac-4bab-aaa9-4db05493a2cf'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:28.027507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] The token cache is being invalidate because project 2f1c4a61fe0e454c805c6e0ed507416f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:28.027976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:28.031140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:28.034971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 851/1728] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:27 2022] DELETE /identity/v3/projects/2f1c4a61fe0e454c805c6e0ed507416f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:28.039998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd770a8-3c31-41a5-a5d6-64a1ec4f3e2b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:28.040497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd770a8-3c31-41a5-a5d6-64a1ec4f3e2b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:28.041005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd770a8-3c31-41a5-a5d6-64a1ec4f3e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:28.062253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:28.067054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:28.118344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6cd770a8-3c31-41a5-a5d6-64a1ec4f3e2b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:28.130331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6cd770a8-3c31-41a5-a5d6-64a1ec4f3e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:28.282816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d21c72-34ac-4bab-aaa9-4db05493a2cf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:28.304145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:28.329766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:28.334504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:28.339193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] PATH_INFO: `/v3/projects/439d6a6b96a7449a83c483989833caf0` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:28.359355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=439d6a6b96a7449a83c483989833caf0)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:28.359355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:28.377520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 878/1729] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:28.377520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1347-cc5e-4091-8c29-425c326a4641 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:28.377520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1347-cc5e-4091-8c29-425c326a4641 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:28.377520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1347-cc5e-4091-8c29-425c326a4641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:28.465305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8ddf1347-cc5e-4091-8c29-425c326a4641 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:28.469594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8ddf1347-cc5e-4091-8c29-425c326a4641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:28.495144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 879/1730] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:28.500067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1347-cc5e-4091-8c29-425c326a4641 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:28.523617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1347-cc5e-4091-8c29-425c326a4641 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:28.527864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '439d6a6b96a7449a83c483989833caf0', 'request_id': 'req-61e99ef2-a604-4d2c-b198-aff210752545'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:28.544283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '439d6a6b96a7449a83c483989833caf0', 'request_id': 'req-61e99ef2-a604-4d2c-b198-aff210752545'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:28.559796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '439d6a6b96a7449a83c483989833caf0', 'request_id': 'req-61e99ef2-a604-4d2c-b198-aff210752545'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:28.588242 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] The token cache is being invalidate because project 439d6a6b96a7449a83c483989833caf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:28.589028 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:28.591275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:28.594700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 852/1731] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:28 2022] DELETE /identity/v3/projects/439d6a6b96a7449a83c483989833caf0 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:28.598800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:28.602658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:28.706975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:28.719740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8ab29-615e-4844-af67-db831b261b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:28.722727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8ab29-615e-4844-af67-db831b261b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:28.727941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8ab29-615e-4844-af67-db831b261b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:28.729694 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8ab29-615e-4844-af67-db831b261b1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:28.730877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8ab29-615e-4844-af67-db831b261b1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:29:28.733792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8ab29-615e-4844-af67-db831b261b1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:28.745247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8ab29-615e-4844-af67-db831b261b1b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:28.815843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 880/1732] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:29:28.822620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8ab29-615e-4844-af67-db831b261b1b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:28.839073 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-60e8ab29-615e-4844-af67-db831b261b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:28.881745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e99ef2-a604-4d2c-b198-aff210752545 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:28.885051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8ab29-615e-4844-af67-db831b261b1b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:28.893603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c6311-8190-4d76-9e15-c08b60969052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:28.893864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec0b31-ed8c-4d90-aff6-bd9b822e390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:28.894490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec0b31-ed8c-4d90-aff6-bd9b822e390d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:28.894606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c6311-8190-4d76-9e15-c08b60969052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:28.894875 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec0b31-ed8c-4d90-aff6-bd9b822e390d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:28.895177 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c6311-8190-4d76-9e15-c08b60969052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:28.895565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c6311-8190-4d76-9e15-c08b60969052 admin admin] PATH_INFO: `/v3/users/111cba50b908499989db6be47b1ff555` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:28.899600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c6311-8190-4d76-9e15-c08b60969052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=111cba50b908499989db6be47b1ff555)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:28.904739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c6311-8190-4d76-9e15-c08b60969052 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:28.907234 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec0b31-ed8c-4d90-aff6-bd9b822e390d admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:28.908079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec0b31-ed8c-4d90-aff6-bd9b822e390d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:28.910974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec0b31-ed8c-4d90-aff6-bd9b822e390d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:28.932833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:28.972010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 881/1733] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:28 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:28.977980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec0b31-ed8c-4d90-aff6-bd9b822e390d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:29.107476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0a3c6311-8190-4d76-9e15-c08b60969052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '111cba50b908499989db6be47b1ff555'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:29.124684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0a3c6311-8190-4d76-9e15-c08b60969052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '111cba50b908499989db6be47b1ff555'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:29.126329 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0a3c6311-8190-4d76-9e15-c08b60969052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '111cba50b908499989db6be47b1ff555'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:29.146088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0a3c6311-8190-4d76-9e15-c08b60969052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '111cba50b908499989db6be47b1ff555'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:29.174687 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0a3c6311-8190-4d76-9e15-c08b60969052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '111cba50b908499989db6be47b1ff555'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:29.188911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 853/1734] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:28 2022] DELETE /identity/v3/users/111cba50b908499989db6be47b1ff555 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:29.259314 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec0b31-ed8c-4d90-aff6-bd9b822e390d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:29.416298 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a56a43-f0bb-4bee-86e1-785847c8d38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:29.416930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a56a43-f0bb-4bee-86e1-785847c8d38d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:29.418021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a56a43-f0bb-4bee-86e1-785847c8d38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:29.418300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a56a43-f0bb-4bee-86e1-785847c8d38d admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:29.419118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a56a43-f0bb-4bee-86e1-785847c8d38d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:29.421122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a56a43-f0bb-4bee-86e1-785847c8d38d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:29.453347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909d56a-86f6-4630-b5ee-24344f964358 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:29.453929 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909d56a-86f6-4630-b5ee-24344f964358 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:29.454479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909d56a-86f6-4630-b5ee-24344f964358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:29.492182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f909d56a-86f6-4630-b5ee-24344f964358 None None] MFA Rules not processed for user `f7aa40f3966d4c97ab6a1a764722969d`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:29.493968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f909d56a-86f6-4630-b5ee-24344f964358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:29.510269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 882/1735] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:28 2022] POST /identity/v3/users => generated 283 bytes in 535 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:29.513006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a56a43-f0bb-4bee-86e1-785847c8d38d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:29.516321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-04a56a43-f0bb-4bee-86e1-785847c8d38d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:29.579071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a56a43-f0bb-4bee-86e1-785847c8d38d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:29.596687 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b838e-e419-440d-8660-59c25ed4d07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:29.597556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b838e-e419-440d-8660-59c25ed4d07d admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:29.598046 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b838e-e419-440d-8660-59c25ed4d07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:29.599010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b838e-e419-440d-8660-59c25ed4d07d admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:29.599819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b838e-e419-440d-8660-59c25ed4d07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:29.601630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b838e-e419-440d-8660-59c25ed4d07d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:29.639528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 883/1736] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:29.641017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b838e-e419-440d-8660-59c25ed4d07d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:29.661831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 854/1737] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:29.669394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b838e-e419-440d-8660-59c25ed4d07d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:29.678427 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d05078-d5e3-4fd2-b997-5a3ed7de3f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:29.680138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d05078-d5e3-4fd2-b997-5a3ed7de3f96 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:29.681894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d05078-d5e3-4fd2-b997-5a3ed7de3f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:29.682324 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d05078-d5e3-4fd2-b997-5a3ed7de3f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:29.683187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d05078-d5e3-4fd2-b997-5a3ed7de3f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:29.686165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d05078-d5e3-4fd2-b997-5a3ed7de3f96 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:29.724693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 884/1738] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:29.726005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909d56a-86f6-4630-b5ee-24344f964358 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:29.751241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909d56a-86f6-4630-b5ee-24344f964358 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:29.763411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f1d53-dda4-471f-b9ff-7171ca4be7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:29.764338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f1d53-dda4-471f-b9ff-7171ca4be7b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:29.764973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f1d53-dda4-471f-b9ff-7171ca4be7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:29.765389 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f1d53-dda4-471f-b9ff-7171ca4be7b5 admin admin] PATH_INFO: `/v3/projects/09896655fd9949bda5169823e3739b28/users/f4790c0bf17a45bd90fd65d3637d8ad4/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:29.836472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f1d53-dda4-471f-b9ff-7171ca4be7b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09896655fd9949bda5169823e3739b28, user_id=f4790c0bf17a45bd90fd65d3637d8ad4, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:29.850271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f1d53-dda4-471f-b9ff-7171ca4be7b5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:29.893900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 855/1739] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:29 2022] PUT /identity/v3/projects/09896655fd9949bda5169823e3739b28/users/f4790c0bf17a45bd90fd65d3637d8ad4/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:29.907305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8d358-a618-4b0d-a38e-1f79b08de31b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:29.907610 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8d358-a618-4b0d-a38e-1f79b08de31b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:29.907776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8d358-a618-4b0d-a38e-1f79b08de31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:30.060510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a0d8d358-a618-4b0d-a38e-1f79b08de31b None None] MFA Rules not processed for user `f4790c0bf17a45bd90fd65d3637d8ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:30.078104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a0d8d358-a618-4b0d-a38e-1f79b08de31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:30.330564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:30.346228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 885/1740] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:29:30.353095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f1d53-dda4-471f-b9ff-7171ca4be7b5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:30.446935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:30.554225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f1d53-dda4-471f-b9ff-7171ca4be7b5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:30.561110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f526d5-7912-4aee-9f3d-d1b84c8a4216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:30.562174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f526d5-7912-4aee-9f3d-d1b84c8a4216 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:30.562522 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f526d5-7912-4aee-9f3d-d1b84c8a4216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:30.563002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f526d5-7912-4aee-9f3d-d1b84c8a4216 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:30.564014 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f526d5-7912-4aee-9f3d-d1b84c8a4216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:30.567534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f526d5-7912-4aee-9f3d-d1b84c8a4216 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:30.628499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 856/1741] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:30 2022] POST /identity/v3/projects => generated 363 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:30.631806 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8d358-a618-4b0d-a38e-1f79b08de31b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:30.648392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8d358-a618-4b0d-a38e-1f79b08de31b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:30.831975 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e15cad-1044-4bf8-810e-b88b7105b2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:30.832986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e15cad-1044-4bf8-810e-b88b7105b2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:30.833489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e15cad-1044-4bf8-810e-b88b7105b2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:30.834239 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e15cad-1044-4bf8-810e-b88b7105b2b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:30.835415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e15cad-1044-4bf8-810e-b88b7105b2b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:30.840805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e15cad-1044-4bf8-810e-b88b7105b2b6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:30.902040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 886/1742] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:30 2022] POST /identity/v3/users => generated 288 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:30.904765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f526d5-7912-4aee-9f3d-d1b84c8a4216 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:30.923269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f526d5-7912-4aee-9f3d-d1b84c8a4216 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:30.942747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980b12c-b326-4988-81fe-f36560842e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:30.950669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980b12c-b326-4988-81fe-f36560842e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:30.951701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980b12c-b326-4988-81fe-f36560842e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:30.954129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980b12c-b326-4988-81fe-f36560842e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:30.955579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980b12c-b326-4988-81fe-f36560842e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:30.961830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980b12c-b326-4988-81fe-f36560842e1c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:31.002035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 857/1743] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:31.006174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e15cad-1044-4bf8-810e-b88b7105b2b6 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:31.018683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e15cad-1044-4bf8-810e-b88b7105b2b6 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:31.040118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2683cb-7e27-4ea9-b251-9df898d65e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:31.040879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2683cb-7e27-4ea9-b251-9df898d65e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:31.041165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2683cb-7e27-4ea9-b251-9df898d65e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:31.041442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2683cb-7e27-4ea9-b251-9df898d65e4b admin admin] PATH_INFO: `/v3/projects/54efd44051f648a1b524dfc4152f500d/users/c78a54ca423542128c7e4655d022fc4f/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:31.073088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2683cb-7e27-4ea9-b251-9df898d65e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=54efd44051f648a1b524dfc4152f500d, user_id=c78a54ca423542128c7e4655d022fc4f, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:31.082002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2683cb-7e27-4ea9-b251-9df898d65e4b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:31.113222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 887/1744] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:31 2022] PUT /identity/v3/projects/54efd44051f648a1b524dfc4152f500d/users/c78a54ca423542128c7e4655d022fc4f/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:31.117700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980b12c-b326-4988-81fe-f36560842e1c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:31.143381 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980b12c-b326-4988-81fe-f36560842e1c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:31.335081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43123a5d-f66e-4ab8-b4ae-6e81a330de90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:31.336743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43123a5d-f66e-4ab8-b4ae-6e81a330de90 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:31.342164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43123a5d-f66e-4ab8-b4ae-6e81a330de90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:31.342612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43123a5d-f66e-4ab8-b4ae-6e81a330de90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:31.343434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43123a5d-f66e-4ab8-b4ae-6e81a330de90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:31.345447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43123a5d-f66e-4ab8-b4ae-6e81a330de90 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:31.373428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 858/1745] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:31.374264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2683cb-7e27-4ea9-b251-9df898d65e4b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:31.407332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2683cb-7e27-4ea9-b251-9df898d65e4b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:31.416818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454858a-2127-4944-86d9-10e164899fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:31.417933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454858a-2127-4944-86d9-10e164899fac admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:31.417933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454858a-2127-4944-86d9-10e164899fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:31.418182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454858a-2127-4944-86d9-10e164899fac admin admin] PATH_INFO: `/v3/projects/54efd44051f648a1b524dfc4152f500d/users/c78a54ca423542128c7e4655d022fc4f/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:31.423810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454858a-2127-4944-86d9-10e164899fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=54efd44051f648a1b524dfc4152f500d, user_id=c78a54ca423542128c7e4655d022fc4f, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:31.426551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454858a-2127-4944-86d9-10e164899fac admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:31.455292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 888/1746] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:31 2022] PUT /identity/v3/projects/54efd44051f648a1b524dfc4152f500d/users/c78a54ca423542128c7e4655d022fc4f/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:31.466979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea518d87-12ca-4745-a440-1947f01e6973 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:31.467463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea518d87-12ca-4745-a440-1947f01e6973 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:31.468013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea518d87-12ca-4745-a440-1947f01e6973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:31.535378 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ea518d87-12ca-4745-a440-1947f01e6973 None None] MFA Rules not processed for user `c78a54ca423542128c7e4655d022fc4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:31.537895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ea518d87-12ca-4745-a440-1947f01e6973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:31.615554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454858a-2127-4944-86d9-10e164899fac admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:31.617675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a454858a-2127-4944-86d9-10e164899fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:31.724908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 859/1747] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:31.741507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea518d87-12ca-4745-a440-1947f01e6973 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:31.746989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ea518d87-12ca-4745-a440-1947f01e6973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:31.776559 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454858a-2127-4944-86d9-10e164899fac admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:31.792666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:31.793524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:31.794165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:31.795356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] PATH_INFO: `/v3/projects/83326596f1ad4c05ae63f7b7954967fa` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:31.796946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=83326596f1ad4c05ae63f7b7954967fa)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:31.798153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:31.806007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:31.877543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea518d87-12ca-4745-a440-1947f01e6973 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:31.889682 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c007f3a-8a19-4608-b598-85e821129cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:31.890318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007f3a-8a19-4608-b598-85e821129cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:31.890595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007f3a-8a19-4608-b598-85e821129cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:31.890868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007f3a-8a19-4608-b598-85e821129cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:31.911234 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83326596f1ad4c05ae63f7b7954967fa', 'request_id': 'req-33c6de68-18c1-4007-9122-657aa9b12e0e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:31.915068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c007f3a-8a19-4608-b598-85e821129cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:31.916396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:31.919635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c007f3a-8a19-4608-b598-85e821129cc5 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:31.930736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83326596f1ad4c05ae63f7b7954967fa', 'request_id': 'req-33c6de68-18c1-4007-9122-657aa9b12e0e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:31.943723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83326596f1ad4c05ae63f7b7954967fa', 'request_id': 'req-33c6de68-18c1-4007-9122-657aa9b12e0e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:31.948729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 860/1748] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:31.970538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] The token cache is being invalidate because project 83326596f1ad4c05ae63f7b7954967fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:31.970780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:31.975397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-33c6de68-18c1-4007-9122-657aa9b12e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:31.980713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 889/1749] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:31 2022] DELETE /identity/v3/projects/83326596f1ad4c05ae63f7b7954967fa => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:29:31.982456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b508bf-76b8-49b7-9d61-e77623f20b44 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:31.982745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b508bf-76b8-49b7-9d61-e77623f20b44 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:31.983111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b508bf-76b8-49b7-9d61-e77623f20b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:31.986079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660df78c-11d1-4d0f-a767-53f4a153ffc7 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:31.986384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660df78c-11d1-4d0f-a767-53f4a153ffc7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:31.986656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660df78c-11d1-4d0f-a767-53f4a153ffc7 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:31.988165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 890/1750] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:29:31.997461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac8915-03c5-4172-9845-2e68d56343b8 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:31.997745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac8915-03c5-4172-9845-2e68d56343b8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:31.998467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac8915-03c5-4172-9845-2e68d56343b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:32.022734 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-49b508bf-76b8-49b7-9d61-e77623f20b44 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:32.023950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-49b508bf-76b8-49b7-9d61-e77623f20b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:32.055972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-74ac8915-03c5-4172-9845-2e68d56343b8 None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:32.060836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-74ac8915-03c5-4172-9845-2e68d56343b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:32.152268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 861/1751] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:32.153716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b508bf-76b8-49b7-9d61-e77623f20b44 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:32.155501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-49b508bf-76b8-49b7-9d61-e77623f20b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:32.242710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 891/1752] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:29:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:29:32.245272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e2509-cdaa-4363-8a99-4161b3949c79 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:32.247143 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e2509-cdaa-4363-8a99-4161b3949c79 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:32.248670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e2509-cdaa-4363-8a99-4161b3949c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:32.314167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b508bf-76b8-49b7-9d61-e77623f20b44 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:32.323035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b06ff7-8c39-4793-abca-c0789d628542 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:32.324029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b06ff7-8c39-4793-abca-c0789d628542 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:32.324523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b06ff7-8c39-4793-abca-c0789d628542 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:32.324856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b06ff7-8c39-4793-abca-c0789d628542 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:32.334493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1b1e2509-cdaa-4363-8a99-4161b3949c79 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:32.336018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1b1e2509-cdaa-4363-8a99-4161b3949c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:32.336636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-49b06ff7-8c39-4793-abca-c0789d628542 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:32.347787 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 892/1753] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:32.352963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e2509-cdaa-4363-8a99-4161b3949c79 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:32.366523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e2509-cdaa-4363-8a99-4161b3949c79 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:32.385446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632eea1-5ac8-4db8-a22d-9ce2b93bd13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:32.386295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632eea1-5ac8-4db8-a22d-9ce2b93bd13b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:32.386674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632eea1-5ac8-4db8-a22d-9ce2b93bd13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:32.387251 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632eea1-5ac8-4db8-a22d-9ce2b93bd13b admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:32.388839 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632eea1-5ac8-4db8-a22d-9ce2b93bd13b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:29:32.389314 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632eea1-5ac8-4db8-a22d-9ce2b93bd13b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:32.391986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632eea1-5ac8-4db8-a22d-9ce2b93bd13b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:32.412241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 893/1754] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:29:32.417205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632eea1-5ac8-4db8-a22d-9ce2b93bd13b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:32.470421 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632eea1-5ac8-4db8-a22d-9ce2b93bd13b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:32.487125 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeffe78-a865-453b-8385-73283f1ef768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:32.500232 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeffe78-a865-453b-8385-73283f1ef768 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:32.500565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeffe78-a865-453b-8385-73283f1ef768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:32.500823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeffe78-a865-453b-8385-73283f1ef768 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:32.501734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeffe78-a865-453b-8385-73283f1ef768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:32.503384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeffe78-a865-453b-8385-73283f1ef768 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:32.549707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b06ff7-8c39-4793-abca-c0789d628542 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:32.553217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b06ff7-8c39-4793-abca-c0789d628542 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:32.579951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 894/1755] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:32 2022] POST /identity/v3/projects => generated 363 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:32.581051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeffe78-a865-453b-8385-73283f1ef768 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:32.584732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cdeffe78-a865-453b-8385-73283f1ef768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:32.591335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 862/1756] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:32.593063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b06ff7-8c39-4793-abca-c0789d628542 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:32.626088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b06ff7-8c39-4793-abca-c0789d628542 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:32.665307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:32.765661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeffe78-a865-453b-8385-73283f1ef768 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:32.777703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec22221-8500-4a54-9470-69f92228fff4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:32.778535 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec22221-8500-4a54-9470-69f92228fff4 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:32.778813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec22221-8500-4a54-9470-69f92228fff4 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:32.779144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec22221-8500-4a54-9470-69f92228fff4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:32.790905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a36c95-83bc-40d6-bda4-fa3cb1dc89a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:32.791437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a36c95-83bc-40d6-bda4-fa3cb1dc89a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:32.791718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a36c95-83bc-40d6-bda4-fa3cb1dc89a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:32.792041 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a36c95-83bc-40d6-bda4-fa3cb1dc89a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:32.792812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a36c95-83bc-40d6-bda4-fa3cb1dc89a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:32.794920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a36c95-83bc-40d6-bda4-fa3cb1dc89a5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:32.796202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec22221-8500-4a54-9470-69f92228fff4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:32.798795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec22221-8500-4a54-9470-69f92228fff4 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:32.825726 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 895/1757] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:32.853074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 863/1758] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:32 2022] POST /identity/v3/users => generated 282 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:32.853641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec22221-8500-4a54-9470-69f92228fff4 service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:32.873481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec22221-8500-4a54-9470-69f92228fff4 service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:32.889031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dde41-fce2-42b8-8fcc-551a18c08f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:32.890813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dde41-fce2-42b8-8fcc-551a18c08f35 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:32.890990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dde41-fce2-42b8-8fcc-551a18c08f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:32.892560 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dde41-fce2-42b8-8fcc-551a18c08f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:32.896239 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279dde41-fce2-42b8-8fcc-551a18c08f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:32.900780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279dde41-fce2-42b8-8fcc-551a18c08f35 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:32.931799 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 896/1759] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:32.933533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a36c95-83bc-40d6-bda4-fa3cb1dc89a5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:32.951759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a36c95-83bc-40d6-bda4-fa3cb1dc89a5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:32.962138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c629e-6d93-4a3c-9b4d-57e00ab20529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:32.964711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c629e-6d93-4a3c-9b4d-57e00ab20529 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:32.965193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c629e-6d93-4a3c-9b4d-57e00ab20529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:32.965476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c629e-6d93-4a3c-9b4d-57e00ab20529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:32.966222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c629e-6d93-4a3c-9b4d-57e00ab20529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:32.967935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c629e-6d93-4a3c-9b4d-57e00ab20529 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:32.995576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 864/1760] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:32.999965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dde41-fce2-42b8-8fcc-551a18c08f35 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:33.007762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:33.013195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dde41-fce2-42b8-8fcc-551a18c08f35 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:33.021967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca43f1-5b29-4436-9183-21d75c6858c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:33.023219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca43f1-5b29-4436-9183-21d75c6858c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:33.023601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca43f1-5b29-4436-9183-21d75c6858c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:33.023999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca43f1-5b29-4436-9183-21d75c6858c0 admin admin] PATH_INFO: `/v3/projects/8195310673fe475284f4dd392746086a/users/cd7c6c9ca9e6419cb0a2cfe8b45cd71e/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:33.043830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca43f1-5b29-4436-9183-21d75c6858c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8195310673fe475284f4dd392746086a, user_id=cd7c6c9ca9e6419cb0a2cfe8b45cd71e, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:33.047754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca43f1-5b29-4436-9183-21d75c6858c0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:33.069046 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 897/1761] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:32 2022] PUT /identity/v3/projects/8195310673fe475284f4dd392746086a/users/cd7c6c9ca9e6419cb0a2cfe8b45cd71e/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:29:33.080938 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f3683-78aa-4237-bb7e-4f314cf144f3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:33.081260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f3683-78aa-4237-bb7e-4f314cf144f3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:33.081536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f3683-78aa-4237-bb7e-4f314cf144f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:33.122958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7e4f3683-78aa-4237-bb7e-4f314cf144f3 None None] MFA Rules not processed for user `cd7c6c9ca9e6419cb0a2cfe8b45cd71e`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:33.124511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7e4f3683-78aa-4237-bb7e-4f314cf144f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:33.209305 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 865/1762] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:33.213997 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca43f1-5b29-4436-9183-21d75c6858c0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:33.283213 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca43f1-5b29-4436-9183-21d75c6858c0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:33.288454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5a14b-950e-403f-8e8d-9c0593dc2251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:33.289065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5a14b-950e-403f-8e8d-9c0593dc2251 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:33.289345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5a14b-950e-403f-8e8d-9c0593dc2251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:33.289615 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5a14b-950e-403f-8e8d-9c0593dc2251 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:33.290240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5a14b-950e-403f-8e8d-9c0593dc2251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:33.292130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5a14b-950e-403f-8e8d-9c0593dc2251 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:33.327788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 898/1763] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:33 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:33.328780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f3683-78aa-4237-bb7e-4f314cf144f3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:33.347117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f3683-78aa-4237-bb7e-4f314cf144f3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:33.433398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154ba48-efab-415c-9bf4-67bb8e4a88c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:33.434883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154ba48-efab-415c-9bf4-67bb8e4a88c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:33.435950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154ba48-efab-415c-9bf4-67bb8e4a88c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:33.436444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154ba48-efab-415c-9bf4-67bb8e4a88c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:33.437266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154ba48-efab-415c-9bf4-67bb8e4a88c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:33.439529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154ba48-efab-415c-9bf4-67bb8e4a88c3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:33.477511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 866/1764] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:33 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:33.478655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5a14b-950e-403f-8e8d-9c0593dc2251 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:33.489926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5a14b-950e-403f-8e8d-9c0593dc2251 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:33.501250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714a39f-1be1-4139-9e43-36f6a1423c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:33.501933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714a39f-1be1-4139-9e43-36f6a1423c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:33.502220 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714a39f-1be1-4139-9e43-36f6a1423c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:33.502494 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714a39f-1be1-4139-9e43-36f6a1423c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:33.503147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714a39f-1be1-4139-9e43-36f6a1423c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:33.504814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714a39f-1be1-4139-9e43-36f6a1423c2b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:33.521075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 899/1765] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:33.525002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154ba48-efab-415c-9bf4-67bb8e4a88c3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:33.565480 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154ba48-efab-415c-9bf4-67bb8e4a88c3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:33.574078 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e606c4-3cab-4a92-828e-034539c26e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:33.575898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e606c4-3cab-4a92-828e-034539c26e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:33.577340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e606c4-3cab-4a92-828e-034539c26e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:33.577965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e606c4-3cab-4a92-828e-034539c26e84 admin admin] PATH_INFO: `/v3/projects/d955699504014a7398174ca13bf90b63/users/868e4e78c46a470e964dcedbb1fb5bfc/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:33.597416 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:33.602792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e606c4-3cab-4a92-828e-034539c26e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d955699504014a7398174ca13bf90b63, user_id=868e4e78c46a470e964dcedbb1fb5bfc, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:33.606858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e606c4-3cab-4a92-828e-034539c26e84 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:33.627575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 867/1766] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:33 2022] PUT /identity/v3/projects/d955699504014a7398174ca13bf90b63/users/868e4e78c46a470e964dcedbb1fb5bfc/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:33.631902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714a39f-1be1-4139-9e43-36f6a1423c2b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:33.645361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714a39f-1be1-4139-9e43-36f6a1423c2b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:33.763831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dede22d-fc13-430e-b7d0-ab8b4e32731a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:33.764547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede22d-fc13-430e-b7d0-ab8b4e32731a admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:33.765040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede22d-fc13-430e-b7d0-ab8b4e32731a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:33.765481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dede22d-fc13-430e-b7d0-ab8b4e32731a admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:33.766200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dede22d-fc13-430e-b7d0-ab8b4e32731a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:33.768061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dede22d-fc13-430e-b7d0-ab8b4e32731a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:33.779251 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 900/1767] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:33.783364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e606c4-3cab-4a92-828e-034539c26e84 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:33.796227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e606c4-3cab-4a92-828e-034539c26e84 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:33.803255 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b27bf-e278-48fb-97ff-5a338cb99c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:33.804264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b27bf-e278-48fb-97ff-5a338cb99c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:33.804794 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b27bf-e278-48fb-97ff-5a338cb99c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:33.805862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b27bf-e278-48fb-97ff-5a338cb99c6d admin admin] PATH_INFO: `/v3/projects/d955699504014a7398174ca13bf90b63/users/868e4e78c46a470e964dcedbb1fb5bfc/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:33.808887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b27bf-e278-48fb-97ff-5a338cb99c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d955699504014a7398174ca13bf90b63, user_id=868e4e78c46a470e964dcedbb1fb5bfc, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:33.811928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b27bf-e278-48fb-97ff-5a338cb99c6d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:33.831866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 868/1768] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:33 2022] PUT /identity/v3/projects/d955699504014a7398174ca13bf90b63/users/868e4e78c46a470e964dcedbb1fb5bfc/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:29:33.842254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7514c-2f87-4c97-a72a-4cfe163d3629 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:33.842636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7514c-2f87-4c97-a72a-4cfe163d3629 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:33.843134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7514c-2f87-4c97-a72a-4cfe163d3629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:33.877077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-22c7514c-2f87-4c97-a72a-4cfe163d3629 None None] MFA Rules not processed for user `868e4e78c46a470e964dcedbb1fb5bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:33.878302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-22c7514c-2f87-4c97-a72a-4cfe163d3629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:33.932713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:33.972644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 901/1769] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:33.981772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b27bf-e278-48fb-97ff-5a338cb99c6d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:33.983727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-331b27bf-e278-48fb-97ff-5a338cb99c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:34.058642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b27bf-e278-48fb-97ff-5a338cb99c6d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:34.062381 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc3024-d687-48d8-a52e-cc5e40f1184b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:34.063017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3024-d687-48d8-a52e-cc5e40f1184b service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:34.063385 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3024-d687-48d8-a52e-cc5e40f1184b service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:34.063820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3024-d687-48d8-a52e-cc5e40f1184b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:34.077212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc3024-d687-48d8-a52e-cc5e40f1184b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:34.079328 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc3024-d687-48d8-a52e-cc5e40f1184b service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:34.094726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 869/1770] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:34.212255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7514c-2f87-4c97-a72a-4cfe163d3629 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:34.222539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7514c-2f87-4c97-a72a-4cfe163d3629 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:34.228267 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64c5bc-5b82-4915-9c80-2a924ce34920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:34.228866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64c5bc-5b82-4915-9c80-2a924ce34920 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:34.229300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64c5bc-5b82-4915-9c80-2a924ce34920 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:34.229576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64c5bc-5b82-4915-9c80-2a924ce34920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:34.313073 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64c5bc-5b82-4915-9c80-2a924ce34920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:34.314753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64c5bc-5b82-4915-9c80-2a924ce34920 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:34.333983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 902/1771] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:37.632906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc3024-d687-48d8-a52e-cc5e40f1184b service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:37.745146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc3024-d687-48d8-a52e-cc5e40f1184b service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:37.749580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:37.750320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:37.752747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:37.753255 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] PATH_INFO: `/v3/users/cd7c6c9ca9e6419cb0a2cfe8b45cd71e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:37.754763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd7c6c9ca9e6419cb0a2cfe8b45cd71e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:37.760850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:37.856977 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd7c6c9ca9e6419cb0a2cfe8b45cd71e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:37.868403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd7c6c9ca9e6419cb0a2cfe8b45cd71e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:37.869188 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd7c6c9ca9e6419cb0a2cfe8b45cd71e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:37.878272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd7c6c9ca9e6419cb0a2cfe8b45cd71e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:37.892822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd7c6c9ca9e6419cb0a2cfe8b45cd71e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:37.895847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:37.902112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 870/1772] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:37 2022] DELETE /identity/v3/users/cd7c6c9ca9e6419cb0a2cfe8b45cd71e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:37.906794 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8685b6-8829-480f-a2e1-b47fa83ef29e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:37.907446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8685b6-8829-480f-a2e1-b47fa83ef29e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:37.907446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8685b6-8829-480f-a2e1-b47fa83ef29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:37.936114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fc8685b6-8829-480f-a2e1-b47fa83ef29e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:37.937658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fc8685b6-8829-480f-a2e1-b47fa83ef29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:38.028691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 903/1773] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:38.045549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:38.052353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:38.143300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfc5a5-fc13-420c-86bb-f786563350fe admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:38.147348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375634a-3090-4188-a188-60a415099bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:38.148360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375634a-3090-4188-a188-60a415099bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:38.148824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375634a-3090-4188-a188-60a415099bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:38.149234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375634a-3090-4188-a188-60a415099bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:38.160237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375634a-3090-4188-a188-60a415099bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:38.162376 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375634a-3090-4188-a188-60a415099bf8 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:38.193317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 871/1774] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:38.719047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8685b6-8829-480f-a2e1-b47fa83ef29e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:38.720712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fc8685b6-8829-480f-a2e1-b47fa83ef29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:38.733791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8685b6-8829-480f-a2e1-b47fa83ef29e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:38.749772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:38.750643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:38.750980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:38.751310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] PATH_INFO: `/v3/projects/8195310673fe475284f4dd392746086a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:38.752808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8195310673fe475284f4dd392746086a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:38.755467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:38.809398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8195310673fe475284f4dd392746086a', 'request_id': 'req-932067e6-e37f-4e95-b635-df1ee1a6cce3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:38.814847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8195310673fe475284f4dd392746086a', 'request_id': 'req-932067e6-e37f-4e95-b635-df1ee1a6cce3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:38.821401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8195310673fe475284f4dd392746086a', 'request_id': 'req-932067e6-e37f-4e95-b635-df1ee1a6cce3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:38.830179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:38.836559 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] The token cache is being invalidate because project 8195310673fe475284f4dd392746086a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:38.837062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:38.838260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:38.839903 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 904/1775] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:38 2022] DELETE /identity/v3/projects/8195310673fe475284f4dd392746086a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:38.843847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375634a-3090-4188-a188-60a415099bf8 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:38.853434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375634a-3090-4188-a188-60a415099bf8 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:38.911146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09f551-e5e1-4538-9300-0248dce53657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:38.911954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09f551-e5e1-4538-9300-0248dce53657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:38.912201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09f551-e5e1-4538-9300-0248dce53657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:38.912498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09f551-e5e1-4538-9300-0248dce53657 admin admin] PATH_INFO: `/v3/users/868e4e78c46a470e964dcedbb1fb5bfc` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:38.913962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09f551-e5e1-4538-9300-0248dce53657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=868e4e78c46a470e964dcedbb1fb5bfc)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:38.915993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09f551-e5e1-4538-9300-0248dce53657 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:38.963693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6a09f551-e5e1-4538-9300-0248dce53657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868e4e78c46a470e964dcedbb1fb5bfc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:38.970569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6a09f551-e5e1-4538-9300-0248dce53657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868e4e78c46a470e964dcedbb1fb5bfc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:38.971246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6a09f551-e5e1-4538-9300-0248dce53657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868e4e78c46a470e964dcedbb1fb5bfc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:38.978156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6a09f551-e5e1-4538-9300-0248dce53657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868e4e78c46a470e964dcedbb1fb5bfc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:38.992748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6a09f551-e5e1-4538-9300-0248dce53657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868e4e78c46a470e964dcedbb1fb5bfc'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:38.998649 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 872/1776] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:38 2022] DELETE /identity/v3/users/868e4e78c46a470e964dcedbb1fb5bfc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:39.292007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:39.293693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:39.308113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932067e6-e37f-4e95-b635-df1ee1a6cce3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:39.376678 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:39.377407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:39.377755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:39.378069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] PATH_INFO: `/v3/projects/d955699504014a7398174ca13bf90b63` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:39.379278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d955699504014a7398174ca13bf90b63)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:39.381336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:39.439597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd955699504014a7398174ca13bf90b63', 'request_id': 'req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:39.447500 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd955699504014a7398174ca13bf90b63', 'request_id': 'req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:39.453609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd955699504014a7398174ca13bf90b63', 'request_id': 'req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:39.467328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] The token cache is being invalidate because project d955699504014a7398174ca13bf90b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:39.467614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:39.468253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8ca0a7ce-2f55-4f26-bbb5-a7b4d75b0fe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:39.469998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 905/1777] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:39 2022] DELETE /identity/v3/projects/d955699504014a7398174ca13bf90b63 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:39.475137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d3851-f7e4-4abc-8380-dba201385c23 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:39.475421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d3851-f7e4-4abc-8380-dba201385c23 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:39.475696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d3851-f7e4-4abc-8380-dba201385c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:39.495971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9e2d3851-f7e4-4abc-8380-dba201385c23 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:39.497200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9e2d3851-f7e4-4abc-8380-dba201385c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:39.509893 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:39.557498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 873/1778] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:39.566390 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e54b59-6db7-46d6-a420-5a526729e475 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:39.567275 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e54b59-6db7-46d6-a420-5a526729e475 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:39.567553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e54b59-6db7-46d6-a420-5a526729e475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:39.589266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-85e54b59-6db7-46d6-a420-5a526729e475 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:39.590489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-85e54b59-6db7-46d6-a420-5a526729e475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:39.614969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 906/1779] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:39.615763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d3851-f7e4-4abc-8380-dba201385c23 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:39.630082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d3851-f7e4-4abc-8380-dba201385c23 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:39.638392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e13885-6ba4-4b0a-ba32-3a588c4f2092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:39.639012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e13885-6ba4-4b0a-ba32-3a588c4f2092 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:39.639345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e13885-6ba4-4b0a-ba32-3a588c4f2092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:39.639608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e13885-6ba4-4b0a-ba32-3a588c4f2092 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:39.640311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e13885-6ba4-4b0a-ba32-3a588c4f2092 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:29:39.640654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e13885-6ba4-4b0a-ba32-3a588c4f2092 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:39.642304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e13885-6ba4-4b0a-ba32-3a588c4f2092 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:39.659673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 874/1780] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:29:39.661352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e54b59-6db7-46d6-a420-5a526729e475 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:39.682715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e54b59-6db7-46d6-a420-5a526729e475 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:39.688124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a856e3-7cb7-4fb4-82bd-37334a7d754c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:39.688865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a856e3-7cb7-4fb4-82bd-37334a7d754c admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:39.689217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a856e3-7cb7-4fb4-82bd-37334a7d754c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:39.690174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a856e3-7cb7-4fb4-82bd-37334a7d754c admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:39.690423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a856e3-7cb7-4fb4-82bd-37334a7d754c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:39.692992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a856e3-7cb7-4fb4-82bd-37334a7d754c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:39.749595 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 907/1781] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:39 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:29:39.755876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e13885-6ba4-4b0a-ba32-3a588c4f2092 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:39.775573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e13885-6ba4-4b0a-ba32-3a588c4f2092 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:39.859954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba0675-cbb0-4ccf-9a78-85336a6d3b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:39.860618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba0675-cbb0-4ccf-9a78-85336a6d3b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:39.860860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba0675-cbb0-4ccf-9a78-85336a6d3b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:39.861127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba0675-cbb0-4ccf-9a78-85336a6d3b7f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:39.861755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba0675-cbb0-4ccf-9a78-85336a6d3b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:39.864409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba0675-cbb0-4ccf-9a78-85336a6d3b7f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:39.887536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 875/1782] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:39 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:39.891977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a856e3-7cb7-4fb4-82bd-37334a7d754c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:39.907579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a856e3-7cb7-4fb4-82bd-37334a7d754c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:39.915033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdf54b-a8d7-455d-b604-5136527d6e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:39.915645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdf54b-a8d7-455d-b604-5136527d6e6a admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:39.915921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdf54b-a8d7-455d-b604-5136527d6e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:39.916215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdf54b-a8d7-455d-b604-5136527d6e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:39.916897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdf54b-a8d7-455d-b604-5136527d6e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:39.918523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdf54b-a8d7-455d-b604-5136527d6e6a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:39.928989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 908/1783] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:39.932738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba0675-cbb0-4ccf-9a78-85336a6d3b7f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:39.943460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba0675-cbb0-4ccf-9a78-85336a6d3b7f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:39.957794 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f6375-4681-4231-be1e-14e3d36eff6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:39.958896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f6375-4681-4231-be1e-14e3d36eff6c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:39.959001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f6375-4681-4231-be1e-14e3d36eff6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:39.959328 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f6375-4681-4231-be1e-14e3d36eff6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:39.960097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f6375-4681-4231-be1e-14e3d36eff6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:39.962014 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f6375-4681-4231-be1e-14e3d36eff6c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:39.979374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 876/1784] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:39.986439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdf54b-a8d7-455d-b604-5136527d6e6a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:39.997120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdf54b-a8d7-455d-b604-5136527d6e6a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:40.001830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4398e41-8c39-4a46-b57f-d185a5f1f3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:40.002482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4398e41-8c39-4a46-b57f-d185a5f1f3ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:40.002759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4398e41-8c39-4a46-b57f-d185a5f1f3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:40.003027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4398e41-8c39-4a46-b57f-d185a5f1f3ab admin admin] PATH_INFO: `/v3/projects/c6a7b8ff2c874f4698d9d61ad75047af/users/a31f2e724dd64d0eb56c27e4f5883fca/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:40.028606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4398e41-8c39-4a46-b57f-d185a5f1f3ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a7b8ff2c874f4698d9d61ad75047af, user_id=a31f2e724dd64d0eb56c27e4f5883fca, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:40.028606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4398e41-8c39-4a46-b57f-d185a5f1f3ab admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:40.050014 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 909/1785] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:39 2022] PUT /identity/v3/projects/c6a7b8ff2c874f4698d9d61ad75047af/users/a31f2e724dd64d0eb56c27e4f5883fca/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:40.061091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fdf96-f6b0-4998-891d-5221a04d3a55 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:40.061380 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fdf96-f6b0-4998-891d-5221a04d3a55 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:40.061662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fdf96-f6b0-4998-891d-5221a04d3a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:40.091565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-443fdf96-f6b0-4998-891d-5221a04d3a55 None None] MFA Rules not processed for user `a31f2e724dd64d0eb56c27e4f5883fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:40.092954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-443fdf96-f6b0-4998-891d-5221a04d3a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:40.155678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 877/1786] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:40.161094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4398e41-8c39-4a46-b57f-d185a5f1f3ab admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:40.197402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:40.226034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4398e41-8c39-4a46-b57f-d185a5f1f3ab admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:40.231400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61608c3a-cdb4-4c5f-b469-40065e71c1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:40.232012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61608c3a-cdb4-4c5f-b469-40065e71c1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:40.232289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61608c3a-cdb4-4c5f-b469-40065e71c1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:40.233945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61608c3a-cdb4-4c5f-b469-40065e71c1d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:40.233945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61608c3a-cdb4-4c5f-b469-40065e71c1d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:40.235638 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61608c3a-cdb4-4c5f-b469-40065e71c1d2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:40.262596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 910/1787] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:40 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:40.266672 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fdf96-f6b0-4998-891d-5221a04d3a55 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:40.278353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fdf96-f6b0-4998-891d-5221a04d3a55 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:40.297489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:40.351862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc8a55-31ce-4ae2-8f4b-d47ccc23d106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:40.352525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc8a55-31ce-4ae2-8f4b-d47ccc23d106 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:40.352814 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc8a55-31ce-4ae2-8f4b-d47ccc23d106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:40.353083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc8a55-31ce-4ae2-8f4b-d47ccc23d106 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:40.353714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc8a55-31ce-4ae2-8f4b-d47ccc23d106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:40.355611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc8a55-31ce-4ae2-8f4b-d47ccc23d106 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:40.378083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 878/1788] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:40 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:29:40.382253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61608c3a-cdb4-4c5f-b469-40065e71c1d2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:40.392758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61608c3a-cdb4-4c5f-b469-40065e71c1d2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:40.402538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89680887-1fd7-4d5e-9f98-33bdcf8f65e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:40.403450 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89680887-1fd7-4d5e-9f98-33bdcf8f65e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:40.403450 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89680887-1fd7-4d5e-9f98-33bdcf8f65e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:40.403600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89680887-1fd7-4d5e-9f98-33bdcf8f65e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:40.404230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89680887-1fd7-4d5e-9f98-33bdcf8f65e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:40.407664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89680887-1fd7-4d5e-9f98-33bdcf8f65e3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:40.418138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 911/1789] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:40.422443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc8a55-31ce-4ae2-8f4b-d47ccc23d106 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:40.432866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc8a55-31ce-4ae2-8f4b-d47ccc23d106 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:40.437501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af3429-1243-4960-9c90-4785bb47e12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:40.438151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af3429-1243-4960-9c90-4785bb47e12e admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:40.438425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af3429-1243-4960-9c90-4785bb47e12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:40.438695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af3429-1243-4960-9c90-4785bb47e12e admin admin] PATH_INFO: `/v3/projects/b43d862ccb404a0ebe2c870da0f70fd2/users/8ea4b645d5424ddca8a7fcbb2ca02961/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:40.454850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af3429-1243-4960-9c90-4785bb47e12e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43d862ccb404a0ebe2c870da0f70fd2, user_id=8ea4b645d5424ddca8a7fcbb2ca02961, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:40.458207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af3429-1243-4960-9c90-4785bb47e12e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:40.471324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 879/1790] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:40 2022] PUT /identity/v3/projects/b43d862ccb404a0ebe2c870da0f70fd2/users/8ea4b645d5424ddca8a7fcbb2ca02961/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:40.475404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89680887-1fd7-4d5e-9f98-33bdcf8f65e3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:40.485576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89680887-1fd7-4d5e-9f98-33bdcf8f65e3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:40.543367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b394f7a-06cd-4930-9436-75199d863397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:40.544069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b394f7a-06cd-4930-9436-75199d863397 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:40.544457 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b394f7a-06cd-4930-9436-75199d863397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:40.544801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b394f7a-06cd-4930-9436-75199d863397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:40.545448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b394f7a-06cd-4930-9436-75199d863397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:40.547168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b394f7a-06cd-4930-9436-75199d863397 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:40.557368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 912/1791] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:40.561676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af3429-1243-4960-9c90-4785bb47e12e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:40.572551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af3429-1243-4960-9c90-4785bb47e12e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:40.578454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a305dd-9d1a-4241-8242-5dd5eba772af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:40.579145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a305dd-9d1a-4241-8242-5dd5eba772af admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:40.579424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a305dd-9d1a-4241-8242-5dd5eba772af admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:40.579697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a305dd-9d1a-4241-8242-5dd5eba772af admin admin] PATH_INFO: `/v3/projects/b43d862ccb404a0ebe2c870da0f70fd2/users/8ea4b645d5424ddca8a7fcbb2ca02961/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:40.581719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a305dd-9d1a-4241-8242-5dd5eba772af admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43d862ccb404a0ebe2c870da0f70fd2, user_id=8ea4b645d5424ddca8a7fcbb2ca02961, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:40.584460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a305dd-9d1a-4241-8242-5dd5eba772af admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:40.595093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 880/1792] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:40 2022] PUT /identity/v3/projects/b43d862ccb404a0ebe2c870da0f70fd2/users/8ea4b645d5424ddca8a7fcbb2ca02961/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:29:40.605664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c6f24-dde2-4158-b415-c1f50d7068ca None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:40.605947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c6f24-dde2-4158-b415-c1f50d7068ca None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:40.606238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c6f24-dde2-4158-b415-c1f50d7068ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:40.633348 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d14c6f24-dde2-4158-b415-c1f50d7068ca None None] MFA Rules not processed for user `8ea4b645d5424ddca8a7fcbb2ca02961`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:40.634529 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d14c6f24-dde2-4158-b415-c1f50d7068ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:40.702251 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 913/1793] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:40.714412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a305dd-9d1a-4241-8242-5dd5eba772af admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:40.716196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b3a305dd-9d1a-4241-8242-5dd5eba772af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:40.776204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a305dd-9d1a-4241-8242-5dd5eba772af admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:40.779995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657e69e-9e47-4757-83c1-69ac7937c530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:40.780722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657e69e-9e47-4757-83c1-69ac7937c530 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:40.780994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657e69e-9e47-4757-83c1-69ac7937c530 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:40.781257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657e69e-9e47-4757-83c1-69ac7937c530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:40.791175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657e69e-9e47-4757-83c1-69ac7937c530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:40.792623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657e69e-9e47-4757-83c1-69ac7937c530 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:40.802491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 881/1794] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:40.831100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c6f24-dde2-4158-b415-c1f50d7068ca None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:40.853188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:40.905568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c6f24-dde2-4158-b415-c1f50d7068ca None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:40.909710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:40.910403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:40.911193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:40.911467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] PATH_INFO: `/v3/users/a31f2e724dd64d0eb56c27e4f5883fca` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:40.912629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a31f2e724dd64d0eb56c27e4f5883fca)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:40.914534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:40.960814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a31f2e724dd64d0eb56c27e4f5883fca'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:40.968336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a31f2e724dd64d0eb56c27e4f5883fca'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:40.969132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a31f2e724dd64d0eb56c27e4f5883fca'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:40.976729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a31f2e724dd64d0eb56c27e4f5883fca'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:40.989646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a31f2e724dd64d0eb56c27e4f5883fca'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:40.995653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 914/1795] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:40 2022] DELETE /identity/v3/users/a31f2e724dd64d0eb56c27e4f5883fca => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:41.000038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9752b6-087d-491d-b6bf-c99763747dd4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:41.000599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9752b6-087d-491d-b6bf-c99763747dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:41.002262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9752b6-087d-491d-b6bf-c99763747dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:41.021074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8c9752b6-087d-491d-b6bf-c99763747dd4 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:41.022288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8c9752b6-087d-491d-b6bf-c99763747dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:41.098985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 882/1796] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:41.108559 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:41.110002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:41.166642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbda0d9-4a11-4b6d-a3b8-ae529b06af5f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:41.170276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba234c-be06-4abe-9412-7dd724425521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:41.170879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba234c-be06-4abe-9412-7dd724425521 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:41.171153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba234c-be06-4abe-9412-7dd724425521 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:41.171423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba234c-be06-4abe-9412-7dd724425521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:41.180767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba234c-be06-4abe-9412-7dd724425521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:41.182172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba234c-be06-4abe-9412-7dd724425521 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:41.191403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 915/1797] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:41.453386 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9752b6-087d-491d-b6bf-c99763747dd4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:41.454910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8c9752b6-087d-491d-b6bf-c99763747dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:41.467407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9752b6-087d-491d-b6bf-c99763747dd4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:41.471966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:41.472642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:41.473475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:41.473749 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] PATH_INFO: `/v3/projects/c6a7b8ff2c874f4698d9d61ad75047af` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:41.475003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a7b8ff2c874f4698d9d61ad75047af)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:41.476951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:41.484181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:41.529351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a7b8ff2c874f4698d9d61ad75047af', 'request_id': 'req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:41.536127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a7b8ff2c874f4698d9d61ad75047af', 'request_id': 'req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:41.542489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a7b8ff2c874f4698d9d61ad75047af', 'request_id': 'req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:41.557842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] The token cache is being invalidate because project c6a7b8ff2c874f4698d9d61ad75047af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:41.558139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:41.558775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:41.560421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 883/1798] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:41 2022] DELETE /identity/v3/projects/c6a7b8ff2c874f4698d9d61ad75047af => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:41.564579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba234c-be06-4abe-9412-7dd724425521 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:41.574329 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba234c-be06-4abe-9412-7dd724425521 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:41.614532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:41.633073 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bda38-5870-4c6e-96dd-cd3e922ebf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:41.633708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bda38-5870-4c6e-96dd-cd3e922ebf61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:41.633986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bda38-5870-4c6e-96dd-cd3e922ebf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:41.634256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bda38-5870-4c6e-96dd-cd3e922ebf61 admin admin] PATH_INFO: `/v3/users/8ea4b645d5424ddca8a7fcbb2ca02961` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:41.635373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38bda38-5870-4c6e-96dd-cd3e922ebf61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ea4b645d5424ddca8a7fcbb2ca02961)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:41.637465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38bda38-5870-4c6e-96dd-cd3e922ebf61 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:41.690756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c38bda38-5870-4c6e-96dd-cd3e922ebf61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ea4b645d5424ddca8a7fcbb2ca02961'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:41.698095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c38bda38-5870-4c6e-96dd-cd3e922ebf61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ea4b645d5424ddca8a7fcbb2ca02961'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:41.698801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c38bda38-5870-4c6e-96dd-cd3e922ebf61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ea4b645d5424ddca8a7fcbb2ca02961'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:41.706082 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c38bda38-5870-4c6e-96dd-cd3e922ebf61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ea4b645d5424ddca8a7fcbb2ca02961'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:41.718482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c38bda38-5870-4c6e-96dd-cd3e922ebf61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ea4b645d5424ddca8a7fcbb2ca02961'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:41.724931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 916/1799] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:41 2022] DELETE /identity/v3/users/8ea4b645d5424ddca8a7fcbb2ca02961 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:42.012659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:42.014248 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:42.027219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910d0f5-c9a0-4619-8852-5b2ea4bc7af2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:42.083844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:42.084533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:42.084817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:42.085092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] PATH_INFO: `/v3/projects/b43d862ccb404a0ebe2c870da0f70fd2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:42.086205 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b43d862ccb404a0ebe2c870da0f70fd2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:42.088147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:42.137337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b43d862ccb404a0ebe2c870da0f70fd2', 'request_id': 'req-3c6265f8-b63e-4a67-85de-bc5439a630a2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:42.144311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b43d862ccb404a0ebe2c870da0f70fd2', 'request_id': 'req-3c6265f8-b63e-4a67-85de-bc5439a630a2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:42.151718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b43d862ccb404a0ebe2c870da0f70fd2', 'request_id': 'req-3c6265f8-b63e-4a67-85de-bc5439a630a2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:42.172244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] The token cache is being invalidate because project b43d862ccb404a0ebe2c870da0f70fd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:42.172561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:42.173207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3c6265f8-b63e-4a67-85de-bc5439a630a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:42.179434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 884/1800] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:42 2022] DELETE /identity/v3/projects/b43d862ccb404a0ebe2c870da0f70fd2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:42.184141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea8385-419f-46d2-aab5-845f185275dc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:42.184504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea8385-419f-46d2-aab5-845f185275dc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:42.184780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea8385-419f-46d2-aab5-845f185275dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:42.209511 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f6ea8385-419f-46d2-aab5-845f185275dc None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:42.211193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f6ea8385-419f-46d2-aab5-845f185275dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:42.271515 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 917/1801] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:42.280107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a712355-970e-4e0d-bac0-478337cd2794 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:42.280421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a712355-970e-4e0d-bac0-478337cd2794 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:42.280700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a712355-970e-4e0d-bac0-478337cd2794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:42.301934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5a712355-970e-4e0d-bac0-478337cd2794 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:42.303111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5a712355-970e-4e0d-bac0-478337cd2794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:42.310989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 885/1802] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:42.315149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea8385-419f-46d2-aab5-845f185275dc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:42.325367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea8385-419f-46d2-aab5-845f185275dc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:42.329908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ed90c-a08f-489c-be33-c4b1d54a05d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:42.330503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ed90c-a08f-489c-be33-c4b1d54a05d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:42.330778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ed90c-a08f-489c-be33-c4b1d54a05d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:42.331054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ed90c-a08f-489c-be33-c4b1d54a05d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:42.331702 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ed90c-a08f-489c-be33-c4b1d54a05d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:29:42.332003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ed90c-a08f-489c-be33-c4b1d54a05d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:42.333658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ed90c-a08f-489c-be33-c4b1d54a05d5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:42.345696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 918/1803] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:29:42.349965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a712355-970e-4e0d-bac0-478337cd2794 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:42.359205 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a712355-970e-4e0d-bac0-478337cd2794 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:42.364387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e65b1c-f5d5-4513-b2d0-808882313d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:42.364998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e65b1c-f5d5-4513-b2d0-808882313d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:42.365277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e65b1c-f5d5-4513-b2d0-808882313d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:42.365578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e65b1c-f5d5-4513-b2d0-808882313d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:42.366210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e65b1c-f5d5-4513-b2d0-808882313d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:42.368167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e65b1c-f5d5-4513-b2d0-808882313d1f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:42.386354 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:42.403396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 886/1804] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:42 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:29:42.407542 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ed90c-a08f-489c-be33-c4b1d54a05d5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:42.417756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ed90c-a08f-489c-be33-c4b1d54a05d5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:42.461406 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:42.475877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1ba3b-91b4-4f5d-b0a4-01208ff9cb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:42.476604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1ba3b-91b4-4f5d-b0a4-01208ff9cb26 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:42.476909 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1ba3b-91b4-4f5d-b0a4-01208ff9cb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:42.477249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1ba3b-91b4-4f5d-b0a4-01208ff9cb26 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:42.477974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1ba3b-91b4-4f5d-b0a4-01208ff9cb26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:42.480058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1ba3b-91b4-4f5d-b0a4-01208ff9cb26 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:42.507273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 919/1805] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:42 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:29:42.508576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e65b1c-f5d5-4513-b2d0-808882313d1f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:42.518897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e65b1c-f5d5-4513-b2d0-808882313d1f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:42.523552 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33eeaa-fe98-45f8-9327-fdcb36cbffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:42.524153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33eeaa-fe98-45f8-9327-fdcb36cbffb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:42.524464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33eeaa-fe98-45f8-9327-fdcb36cbffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:42.524741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33eeaa-fe98-45f8-9327-fdcb36cbffb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:42.525382 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33eeaa-fe98-45f8-9327-fdcb36cbffb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:42.527010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33eeaa-fe98-45f8-9327-fdcb36cbffb3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:42.537372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 887/1806] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:29:42.541326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1ba3b-91b4-4f5d-b0a4-01208ff9cb26 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:42.551840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1ba3b-91b4-4f5d-b0a4-01208ff9cb26 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:42.556413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151a6e1-9e46-4e17-85dc-598df5ca56a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:42.557026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151a6e1-9e46-4e17-85dc-598df5ca56a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:42.557305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151a6e1-9e46-4e17-85dc-598df5ca56a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:42.557578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151a6e1-9e46-4e17-85dc-598df5ca56a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:42.558223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151a6e1-9e46-4e17-85dc-598df5ca56a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:42.559805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151a6e1-9e46-4e17-85dc-598df5ca56a8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:42.571625 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 920/1807] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:29:42.576633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33eeaa-fe98-45f8-9327-fdcb36cbffb3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:42.587636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33eeaa-fe98-45f8-9327-fdcb36cbffb3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:42.592861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca796b3-977d-4bd4-92d0-2b40a8e60317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:42.593518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca796b3-977d-4bd4-92d0-2b40a8e60317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:42.593797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca796b3-977d-4bd4-92d0-2b40a8e60317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:42.594071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca796b3-977d-4bd4-92d0-2b40a8e60317 admin admin] PATH_INFO: `/v3/projects/3054b783f4d04c6ca4f61fa1cc807905/users/76f38897be074678a623860affef8a4f/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:42.609560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca796b3-977d-4bd4-92d0-2b40a8e60317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3054b783f4d04c6ca4f61fa1cc807905, user_id=76f38897be074678a623860affef8a4f, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:42.612224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca796b3-977d-4bd4-92d0-2b40a8e60317 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:42.625657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 888/1808] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:42 2022] PUT /identity/v3/projects/3054b783f4d04c6ca4f61fa1cc807905/users/76f38897be074678a623860affef8a4f/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:42.636301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af74bf4-9199-4f62-be05-50507dfa88da None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:42.636636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af74bf4-9199-4f62-be05-50507dfa88da None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:42.636907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af74bf4-9199-4f62-be05-50507dfa88da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:42.665327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8af74bf4-9199-4f62-be05-50507dfa88da None None] MFA Rules not processed for user `76f38897be074678a623860affef8a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:42.666527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8af74bf4-9199-4f62-be05-50507dfa88da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:42.735732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 921/1809] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:42.747818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca796b3-977d-4bd4-92d0-2b40a8e60317 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:42.749541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8ca796b3-977d-4bd4-92d0-2b40a8e60317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:42.843839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca796b3-977d-4bd4-92d0-2b40a8e60317 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:42.848258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252ee11-adfb-4215-a350-ec6c598492ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a9b371434243daa4176215daefb79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:42.848890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252ee11-adfb-4215-a350-ec6c598492ff service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:42.849165 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252ee11-adfb-4215-a350-ec6c598492ff service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:42.849434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252ee11-adfb-4215-a350-ec6c598492ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:42.862210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252ee11-adfb-4215-a350-ec6c598492ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:42.863883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252ee11-adfb-4215-a350-ec6c598492ff service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:42.876823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 889/1810] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:44.617077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a2b37-daa9-418a-b2de-f2662bf4f95c None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:44.617660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a2b37-daa9-418a-b2de-f2662bf4f95c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:44.617845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a2b37-daa9-418a-b2de-f2662bf4f95c None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:44.619303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 922/1811] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:29:44.642435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfae851-fcd4-4e4b-aa52-4f11c2407a49 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:44.644401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfae851-fcd4-4e4b-aa52-4f11c2407a49 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:44.645511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfae851-fcd4-4e4b-aa52-4f11c2407a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:44.697471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dbfae851-fcd4-4e4b-aa52-4f11c2407a49 None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:44.698971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dbfae851-fcd4-4e4b-aa52-4f11c2407a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:44.896944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 890/1812] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:29:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:44.925652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a2b37-daa9-418a-b2de-f2662bf4f95c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:44.927778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e93a2b37-daa9-418a-b2de-f2662bf4f95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:44.970136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a2b37-daa9-418a-b2de-f2662bf4f95c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:44.989521 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcad2bb-ee64-4362-8aff-86af815ba6c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:44.990344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcad2bb-ee64-4362-8aff-86af815ba6c5 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:44.990689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcad2bb-ee64-4362-8aff-86af815ba6c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:44.991031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcad2bb-ee64-4362-8aff-86af815ba6c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:45.010113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcad2bb-ee64-4362-8aff-86af815ba6c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:45.014973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcad2bb-ee64-4362-8aff-86af815ba6c5 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:45.084414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 923/1813] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:47.811785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfae851-fcd4-4e4b-aa52-4f11c2407a49 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:47.813231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dbfae851-fcd4-4e4b-aa52-4f11c2407a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:47.851584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:47.883300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfae851-fcd4-4e4b-aa52-4f11c2407a49 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:47.887970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:47.888671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:47.889102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:47.889381 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] PATH_INFO: `/v3/users/f4790c0bf17a45bd90fd65d3637d8ad4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:47.890573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4790c0bf17a45bd90fd65d3637d8ad4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:47.892679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:47.971141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4790c0bf17a45bd90fd65d3637d8ad4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:47.979602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4790c0bf17a45bd90fd65d3637d8ad4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:47.980298 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4790c0bf17a45bd90fd65d3637d8ad4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:47.988474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4790c0bf17a45bd90fd65d3637d8ad4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:48.005750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4790c0bf17a45bd90fd65d3637d8ad4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:48.015090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 891/1814] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:47 2022] DELETE /identity/v3/users/f4790c0bf17a45bd90fd65d3637d8ad4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:48.019967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3f4a2-8273-4ffd-9739-bcfb5b296591 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:48.020581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3f4a2-8273-4ffd-9739-bcfb5b296591 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:48.020963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3f4a2-8273-4ffd-9739-bcfb5b296591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:48.043858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7fd3f4a2-8273-4ffd-9739-bcfb5b296591 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:48.045358 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7fd3f4a2-8273-4ffd-9739-bcfb5b296591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:48.148619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 924/1815] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:48.156101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:48.157709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:48.221120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6968f4-9580-49c2-b0a2-19653ca7472d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:48.225488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eff875-e9ae-478d-a5b3-247c391d08a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:48.226092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eff875-e9ae-478d-a5b3-247c391d08a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:48.226365 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eff875-e9ae-478d-a5b3-247c391d08a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:48.226633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eff875-e9ae-478d-a5b3-247c391d08a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:48.235890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eff875-e9ae-478d-a5b3-247c391d08a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:48.237588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eff875-e9ae-478d-a5b3-247c391d08a6 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:48.256551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 892/1816] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:48.740104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3f4a2-8273-4ffd-9739-bcfb5b296591 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:48.743216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7fd3f4a2-8273-4ffd-9739-bcfb5b296591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:48.759337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3f4a2-8273-4ffd-9739-bcfb5b296591 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:48.764601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:48.765240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:48.765514 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:48.765838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] PATH_INFO: `/v3/projects/09896655fd9949bda5169823e3739b28` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:48.766883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09896655fd9949bda5169823e3739b28)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:48.768798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:48.808105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:48.817260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09896655fd9949bda5169823e3739b28', 'request_id': 'req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:48.823966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09896655fd9949bda5169823e3739b28', 'request_id': 'req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:48.830228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09896655fd9949bda5169823e3739b28', 'request_id': 'req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:48.842039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] The token cache is being invalidate because project 09896655fd9949bda5169823e3739b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:48.842323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:48.842955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:48.844529 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 925/1817] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:48 2022] DELETE /identity/v3/projects/09896655fd9949bda5169823e3739b28 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:48.849028 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eff875-e9ae-478d-a5b3-247c391d08a6 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:48.858530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eff875-e9ae-478d-a5b3-247c391d08a6 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:48.910743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:48.917514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63eac9-6498-40dd-be2f-1b4ba02242a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:48.918151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63eac9-6498-40dd-be2f-1b4ba02242a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:48.918765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63eac9-6498-40dd-be2f-1b4ba02242a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:48.919238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63eac9-6498-40dd-be2f-1b4ba02242a6 admin admin] PATH_INFO: `/v3/users/c78a54ca423542128c7e4655d022fc4f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:48.920555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63eac9-6498-40dd-be2f-1b4ba02242a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c78a54ca423542128c7e4655d022fc4f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:48.922966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63eac9-6498-40dd-be2f-1b4ba02242a6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:48.971478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4f63eac9-6498-40dd-be2f-1b4ba02242a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c78a54ca423542128c7e4655d022fc4f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:48.978311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4f63eac9-6498-40dd-be2f-1b4ba02242a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c78a54ca423542128c7e4655d022fc4f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:48.978968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4f63eac9-6498-40dd-be2f-1b4ba02242a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c78a54ca423542128c7e4655d022fc4f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:48.986516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4f63eac9-6498-40dd-be2f-1b4ba02242a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c78a54ca423542128c7e4655d022fc4f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:48.999818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4f63eac9-6498-40dd-be2f-1b4ba02242a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c78a54ca423542128c7e4655d022fc4f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:49.014738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 893/1818] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:48 2022] DELETE /identity/v3/users/c78a54ca423542128c7e4655d022fc4f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:49.427464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:49.429469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:49.447892 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a947a5f-ab50-42fa-bfd2-2503ad3a2a58 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:49.513463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:49.514193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:49.514547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:49.514914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] PATH_INFO: `/v3/projects/54efd44051f648a1b524dfc4152f500d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:49.516112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54efd44051f648a1b524dfc4152f500d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:49.518158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:49.578995 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54efd44051f648a1b524dfc4152f500d', 'request_id': 'req-5337f4ce-c366-45c1-ac78-293fdc103991'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:49.589238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54efd44051f648a1b524dfc4152f500d', 'request_id': 'req-5337f4ce-c366-45c1-ac78-293fdc103991'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:49.597762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54efd44051f648a1b524dfc4152f500d', 'request_id': 'req-5337f4ce-c366-45c1-ac78-293fdc103991'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:49.620910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] The token cache is being invalidate because project 54efd44051f648a1b524dfc4152f500d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:49.621497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:49.624887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5337f4ce-c366-45c1-ac78-293fdc103991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:49.627608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 926/1819] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:49 2022] DELETE /identity/v3/projects/54efd44051f648a1b524dfc4152f500d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:49.635238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7950434-aad1-4f26-9601-ccf028da6555 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:49.635532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7950434-aad1-4f26-9601-ccf028da6555 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:49.635810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7950434-aad1-4f26-9601-ccf028da6555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:49.659837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f7950434-aad1-4f26-9601-ccf028da6555 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:49.661611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f7950434-aad1-4f26-9601-ccf028da6555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:49.767340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 894/1820] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:49.777302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66f67d-79cc-4a95-8af9-b6c5900a9bec None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:49.778396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66f67d-79cc-4a95-8af9-b6c5900a9bec None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:49.779428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66f67d-79cc-4a95-8af9-b6c5900a9bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:49.842451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2e66f67d-79cc-4a95-8af9-b6c5900a9bec None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:49.844598 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2e66f67d-79cc-4a95-8af9-b6c5900a9bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:49.855788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 927/1821] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:49.860541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7950434-aad1-4f26-9601-ccf028da6555 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:49.873557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7950434-aad1-4f26-9601-ccf028da6555 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:49.880949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01923a-166a-4f77-a1a5-04f985a5f213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:49.881910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01923a-166a-4f77-a1a5-04f985a5f213 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:49.882463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01923a-166a-4f77-a1a5-04f985a5f213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:49.882911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01923a-166a-4f77-a1a5-04f985a5f213 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:49.883740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01923a-166a-4f77-a1a5-04f985a5f213 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:29:49.884261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01923a-166a-4f77-a1a5-04f985a5f213 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:49.886154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01923a-166a-4f77-a1a5-04f985a5f213 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:49.915326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 895/1822] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:29:49.918439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66f67d-79cc-4a95-8af9-b6c5900a9bec None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:49.928756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66f67d-79cc-4a95-8af9-b6c5900a9bec None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:49.933449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff1fa2-314b-4825-8dbe-d08af0158b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:49.934060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff1fa2-314b-4825-8dbe-d08af0158b1c admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:49.934331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff1fa2-314b-4825-8dbe-d08af0158b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:49.934596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff1fa2-314b-4825-8dbe-d08af0158b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:49.935240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff1fa2-314b-4825-8dbe-d08af0158b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:49.937258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff1fa2-314b-4825-8dbe-d08af0158b1c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:50.012000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01923a-166a-4f77-a1a5-04f985a5f213 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:50.022969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 928/1823] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:49 2022] POST /identity/v3/projects => generated 399 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:50.024815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01923a-166a-4f77-a1a5-04f985a5f213 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:50.131989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fb557-3adf-4b52-b207-0258fc9f98d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:50.132737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fb557-3adf-4b52-b207-0258fc9f98d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:50.133058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fb557-3adf-4b52-b207-0258fc9f98d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:50.133814 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fb557-3adf-4b52-b207-0258fc9f98d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:50.134181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fb557-3adf-4b52-b207-0258fc9f98d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:50.136964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fb557-3adf-4b52-b207-0258fc9f98d6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:50.149684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:50.165313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 896/1824] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:50 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:50.169044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff1fa2-314b-4825-8dbe-d08af0158b1c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:50.181330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff1fa2-314b-4825-8dbe-d08af0158b1c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:50.186269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2e139-a688-48e8-b1db-222fe230f27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:50.186942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2e139-a688-48e8-b1db-222fe230f27d admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:50.187300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2e139-a688-48e8-b1db-222fe230f27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:50.187641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2e139-a688-48e8-b1db-222fe230f27d admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:50.188404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2e139-a688-48e8-b1db-222fe230f27d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:50.190059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2e139-a688-48e8-b1db-222fe230f27d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:50.201449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 929/1825] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:50.205494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fb557-3adf-4b52-b207-0258fc9f98d6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:50.216241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fb557-3adf-4b52-b207-0258fc9f98d6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:50.220907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a6ea2-cc09-49b1-814a-07d5950f8716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:50.221552 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a6ea2-cc09-49b1-814a-07d5950f8716 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:50.221832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a6ea2-cc09-49b1-814a-07d5950f8716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:50.222104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a6ea2-cc09-49b1-814a-07d5950f8716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:50.222777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a6ea2-cc09-49b1-814a-07d5950f8716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:50.224390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a6ea2-cc09-49b1-814a-07d5950f8716 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:50.235087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 897/1826] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:29:50.239386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2e139-a688-48e8-b1db-222fe230f27d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:50.249308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2e139-a688-48e8-b1db-222fe230f27d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:50.254386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2090250-42ff-4447-855f-4b13970e0f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:50.255033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2090250-42ff-4447-855f-4b13970e0f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:50.255309 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2090250-42ff-4447-855f-4b13970e0f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:50.255577 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2090250-42ff-4447-855f-4b13970e0f12 admin admin] PATH_INFO: `/v3/projects/2369ff1400f543029a95d4cd579b109d/users/4a455a2a213b49c797b89971a85c6173/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:50.270701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2090250-42ff-4447-855f-4b13970e0f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2369ff1400f543029a95d4cd579b109d, user_id=4a455a2a213b49c797b89971a85c6173, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:50.273156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2090250-42ff-4447-855f-4b13970e0f12 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:50.275375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:50.294126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 930/1827] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:50 2022] PUT /identity/v3/projects/2369ff1400f543029a95d4cd579b109d/users/4a455a2a213b49c797b89971a85c6173/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:50.305547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf6248-bd07-47d5-ac85-66f537c7d046 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:50.305804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf6248-bd07-47d5-ac85-66f537c7d046 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:50.306188 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf6248-bd07-47d5-ac85-66f537c7d046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:50.339403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9aaf6248-bd07-47d5-ac85-66f537c7d046 None None] MFA Rules not processed for user `4a455a2a213b49c797b89971a85c6173`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:50.340888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9aaf6248-bd07-47d5-ac85-66f537c7d046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:50.441531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 898/1828] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:50 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:50.445589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2090250-42ff-4447-855f-4b13970e0f12 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:50.527863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2090250-42ff-4447-855f-4b13970e0f12 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:50.532917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a4e5a-f459-4751-9494-30d24f1d1231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:50.533621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a4e5a-f459-4751-9494-30d24f1d1231 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:50.533965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a4e5a-f459-4751-9494-30d24f1d1231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:50.534497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a4e5a-f459-4751-9494-30d24f1d1231 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:50.535254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a4e5a-f459-4751-9494-30d24f1d1231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:50.537188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a4e5a-f459-4751-9494-30d24f1d1231 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:50.565188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 931/1829] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:50 2022] POST /identity/v3/projects => generated 401 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:50.569860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf6248-bd07-47d5-ac85-66f537c7d046 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:50.587792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf6248-bd07-47d5-ac85-66f537c7d046 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:50.657532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2505411-0905-4090-91a8-4e3f83c781f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:50.658341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2505411-0905-4090-91a8-4e3f83c781f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:50.658624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2505411-0905-4090-91a8-4e3f83c781f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:50.658898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2505411-0905-4090-91a8-4e3f83c781f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:50.660084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2505411-0905-4090-91a8-4e3f83c781f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:50.662564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2505411-0905-4090-91a8-4e3f83c781f8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:50.701670 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 899/1830] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:50 2022] POST /identity/v3/users => generated 307 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:50.708433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a4e5a-f459-4751-9494-30d24f1d1231 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:50.719455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a4e5a-f459-4751-9494-30d24f1d1231 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:50.724785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b0487-41e8-483f-bb92-305f2df1f192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:50.725624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b0487-41e8-483f-bb92-305f2df1f192 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:50.725908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b0487-41e8-483f-bb92-305f2df1f192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:50.726180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b0487-41e8-483f-bb92-305f2df1f192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:50.726846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b0487-41e8-483f-bb92-305f2df1f192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:50.728601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b0487-41e8-483f-bb92-305f2df1f192 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:50.744166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 932/1831] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:50.748862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2505411-0905-4090-91a8-4e3f83c781f8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:50.761975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2505411-0905-4090-91a8-4e3f83c781f8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:50.778357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f2c6e-fff1-40c2-b4dc-c5b26db0114e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:50.779489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f2c6e-fff1-40c2-b4dc-c5b26db0114e admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:50.779920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f2c6e-fff1-40c2-b4dc-c5b26db0114e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:50.780352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f2c6e-fff1-40c2-b4dc-c5b26db0114e admin admin] PATH_INFO: `/v3/projects/8a5f10c3b834403d96c0efd92149e7b9/users/60aa3ad2be024831b2640cc5286e3906/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:50.806900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f2c6e-fff1-40c2-b4dc-c5b26db0114e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5f10c3b834403d96c0efd92149e7b9, user_id=60aa3ad2be024831b2640cc5286e3906, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:50.810268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f2c6e-fff1-40c2-b4dc-c5b26db0114e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:50.831700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 900/1832] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:50 2022] PUT /identity/v3/projects/8a5f10c3b834403d96c0efd92149e7b9/users/60aa3ad2be024831b2640cc5286e3906/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:29:50.837763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b0487-41e8-483f-bb92-305f2df1f192 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:50.850550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b0487-41e8-483f-bb92-305f2df1f192 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:50.915800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262384a-7384-4de2-a8de-157d7d7c86ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:50.916516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262384a-7384-4de2-a8de-157d7d7c86ae admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:50.916800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262384a-7384-4de2-a8de-157d7d7c86ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:50.917096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262384a-7384-4de2-a8de-157d7d7c86ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:50.917845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262384a-7384-4de2-a8de-157d7d7c86ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:50.919540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262384a-7384-4de2-a8de-157d7d7c86ae admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:50.934794 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 933/1833] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:50.939181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f2c6e-fff1-40c2-b4dc-c5b26db0114e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:50.958752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f2c6e-fff1-40c2-b4dc-c5b26db0114e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:50.970888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d32a2-af0d-482e-a5a6-f8597d573307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:50.971718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d32a2-af0d-482e-a5a6-f8597d573307 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:50.972104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d32a2-af0d-482e-a5a6-f8597d573307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:50.973094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d32a2-af0d-482e-a5a6-f8597d573307 admin admin] PATH_INFO: `/v3/projects/8a5f10c3b834403d96c0efd92149e7b9/users/60aa3ad2be024831b2640cc5286e3906/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:50.975502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d32a2-af0d-482e-a5a6-f8597d573307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5f10c3b834403d96c0efd92149e7b9, user_id=60aa3ad2be024831b2640cc5286e3906, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:50.978623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d32a2-af0d-482e-a5a6-f8597d573307 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:50.996719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 901/1834] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:50 2022] PUT /identity/v3/projects/8a5f10c3b834403d96c0efd92149e7b9/users/60aa3ad2be024831b2640cc5286e3906/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:51.008647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39f6f5-cc1f-4915-b135-50ddae11a432 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:51.008936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39f6f5-cc1f-4915-b135-50ddae11a432 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:51.009265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39f6f5-cc1f-4915-b135-50ddae11a432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:51.079337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1e39f6f5-cc1f-4915-b135-50ddae11a432 None None] MFA Rules not processed for user `60aa3ad2be024831b2640cc5286e3906`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:51.083271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1e39f6f5-cc1f-4915-b135-50ddae11a432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:51.235148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:51.263757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 934/1835] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:51 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:51.271197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d32a2-af0d-482e-a5a6-f8597d573307 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:51.278603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cb6d32a2-af0d-482e-a5a6-f8597d573307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:51.308249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:51.340875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d32a2-af0d-482e-a5a6-f8597d573307 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:51.345003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e9913-a815-461a-8f51-35c21238ac30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:51.345622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e9913-a815-461a-8f51-35c21238ac30 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:51.345896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e9913-a815-461a-8f51-35c21238ac30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:51.346163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e9913-a815-461a-8f51-35c21238ac30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:51.355919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e9913-a815-461a-8f51-35c21238ac30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:51.357802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e9913-a815-461a-8f51-35c21238ac30 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:51.368648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 902/1836] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:51 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:51.455940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39f6f5-cc1f-4915-b135-50ddae11a432 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:51.471695 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39f6f5-cc1f-4915-b135-50ddae11a432 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:51.480196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625ab38-cf53-45d1-8aeb-8ed3dbe83919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:51.481095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625ab38-cf53-45d1-8aeb-8ed3dbe83919 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:51.481442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625ab38-cf53-45d1-8aeb-8ed3dbe83919 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:51.481806 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625ab38-cf53-45d1-8aeb-8ed3dbe83919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:51.564562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625ab38-cf53-45d1-8aeb-8ed3dbe83919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:51.566017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625ab38-cf53-45d1-8aeb-8ed3dbe83919 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:51.576505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 935/1837] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:51 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:52.565474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786c776-609c-496a-a88a-336f13af67a3 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:52.566038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786c776-609c-496a-a88a-336f13af67a3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:52.566038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786c776-609c-496a-a88a-336f13af67a3 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:52.567900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 903/1838] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:29:52.577518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f9fd5-f566-413e-a7be-21418c3bcbfe None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:52.577958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f9fd5-f566-413e-a7be-21418c3bcbfe None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:52.578086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f9fd5-f566-413e-a7be-21418c3bcbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:52.600023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d18f9fd5-f566-413e-a7be-21418c3bcbfe None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:52.601434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d18f9fd5-f566-413e-a7be-21418c3bcbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:52.681791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 936/1839] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:29:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:52.704491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786c776-609c-496a-a88a-336f13af67a3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:52.706568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8786c776-609c-496a-a88a-336f13af67a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:52.725833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786c776-609c-496a-a88a-336f13af67a3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:52.733728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f551863-2a6f-4f8d-87c3-a24d2fc77cb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:52.734367 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f551863-2a6f-4f8d-87c3-a24d2fc77cb9 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:52.734625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f551863-2a6f-4f8d-87c3-a24d2fc77cb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:52.734909 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f551863-2a6f-4f8d-87c3-a24d2fc77cb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:52.745627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f551863-2a6f-4f8d-87c3-a24d2fc77cb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:52.747210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f551863-2a6f-4f8d-87c3-a24d2fc77cb9 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:52.758837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 904/1840] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:57.010689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f9fd5-f566-413e-a7be-21418c3bcbfe None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:57.074039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f9fd5-f566-413e-a7be-21418c3bcbfe None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:57.078802 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:57.079437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:57.079710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:57.079975 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] PATH_INFO: `/v3/users/4a455a2a213b49c797b89971a85c6173` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:57.081104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a455a2a213b49c797b89971a85c6173)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:57.083056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:57.110530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:57.146392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a455a2a213b49c797b89971a85c6173'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:57.153819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a455a2a213b49c797b89971a85c6173'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:57.154489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a455a2a213b49c797b89971a85c6173'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:57.163613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a455a2a213b49c797b89971a85c6173'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:57.176992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a455a2a213b49c797b89971a85c6173'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:57.182917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 937/1841] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:57 2022] DELETE /identity/v3/users/4a455a2a213b49c797b89971a85c6173 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:57.187446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36485e01-6e84-4042-9127-4df1704d9e1e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:57.188012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36485e01-6e84-4042-9127-4df1704d9e1e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:57.188288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36485e01-6e84-4042-9127-4df1704d9e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:57.208173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-36485e01-6e84-4042-9127-4df1704d9e1e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:57.209662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-36485e01-6e84-4042-9127-4df1704d9e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:57.271778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 905/1842] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:57.281429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:57.282877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:57.341127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf7b44-9ed7-4aad-a1ba-99e04bc6a118 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:57.345152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0a3e6-08dc-4d45-9dc4-c9d0ea8dff99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:57.345780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0a3e6-08dc-4d45-9dc4-c9d0ea8dff99 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:57.346079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0a3e6-08dc-4d45-9dc4-c9d0ea8dff99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:57.346379 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0a3e6-08dc-4d45-9dc4-c9d0ea8dff99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:57.357187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0a3e6-08dc-4d45-9dc4-c9d0ea8dff99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:57.358670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0a3e6-08dc-4d45-9dc4-c9d0ea8dff99 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:57.368472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 938/1843] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:29:57.714058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36485e01-6e84-4042-9127-4df1704d9e1e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:57.715774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-36485e01-6e84-4042-9127-4df1704d9e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:57.744173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36485e01-6e84-4042-9127-4df1704d9e1e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:57.749223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:57.749803 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:57.750082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:57.750445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] PATH_INFO: `/v3/projects/2369ff1400f543029a95d4cd579b109d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:57.751453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2369ff1400f543029a95d4cd579b109d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:57.755197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:57.827334 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2369ff1400f543029a95d4cd579b109d', 'request_id': 'req-ee64403e-54f6-4ff0-9741-76d48e410f79'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:57.834217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2369ff1400f543029a95d4cd579b109d', 'request_id': 'req-ee64403e-54f6-4ff0-9741-76d48e410f79'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:57.842075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2369ff1400f543029a95d4cd579b109d', 'request_id': 'req-ee64403e-54f6-4ff0-9741-76d48e410f79'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:57.858527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] The token cache is being invalidate because project 2369ff1400f543029a95d4cd579b109d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:57.859268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:57.861047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:57.863633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 906/1844] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:57 2022] DELETE /identity/v3/projects/2369ff1400f543029a95d4cd579b109d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:57.867460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0a3e6-08dc-4d45-9dc4-c9d0ea8dff99 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:57.878082 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0a3e6-08dc-4d45-9dc4-c9d0ea8dff99 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:57.941588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4108a-51c3-4821-8190-70e8541d9b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:57.942223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4108a-51c3-4821-8190-70e8541d9b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:57.942507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4108a-51c3-4821-8190-70e8541d9b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:57.942776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4108a-51c3-4821-8190-70e8541d9b75 admin admin] PATH_INFO: `/v3/users/60aa3ad2be024831b2640cc5286e3906` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:57.943942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4108a-51c3-4821-8190-70e8541d9b75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60aa3ad2be024831b2640cc5286e3906)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:57.945867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4108a-51c3-4821-8190-70e8541d9b75 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:57.990445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:57.993125 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-98e4108a-51c3-4821-8190-70e8541d9b75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60aa3ad2be024831b2640cc5286e3906'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:58.003312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-98e4108a-51c3-4821-8190-70e8541d9b75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60aa3ad2be024831b2640cc5286e3906'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:58.004039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-98e4108a-51c3-4821-8190-70e8541d9b75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60aa3ad2be024831b2640cc5286e3906'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:58.011971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-98e4108a-51c3-4821-8190-70e8541d9b75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60aa3ad2be024831b2640cc5286e3906'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:58.025446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-98e4108a-51c3-4821-8190-70e8541d9b75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60aa3ad2be024831b2640cc5286e3906'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:58.031424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 939/1845] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:29:57 2022] DELETE /identity/v3/users/60aa3ad2be024831b2640cc5286e3906 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:29:58.370837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:58.375217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:58.391539 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64403e-54f6-4ff0-9741-76d48e410f79 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:58.452573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:58.480544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:58.481295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:58.481637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:58.481979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] PATH_INFO: `/v3/projects/8a5f10c3b834403d96c0efd92149e7b9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:58.483241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5f10c3b834403d96c0efd92149e7b9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:58.485514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:58.535499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5f10c3b834403d96c0efd92149e7b9', 'request_id': 'req-75bf157d-97f5-49db-b0f2-4df122201bfa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:58.541863 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5f10c3b834403d96c0efd92149e7b9', 'request_id': 'req-75bf157d-97f5-49db-b0f2-4df122201bfa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:58.548473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5f10c3b834403d96c0efd92149e7b9', 'request_id': 'req-75bf157d-97f5-49db-b0f2-4df122201bfa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:58.561811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] The token cache is being invalidate because project 8a5f10c3b834403d96c0efd92149e7b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:29:58.562108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:58.562748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-75bf157d-97f5-49db-b0f2-4df122201bfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:29:58.564310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 907/1846] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:29:58 2022] DELETE /identity/v3/projects/8a5f10c3b834403d96c0efd92149e7b9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:58.569219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20436f96-64b7-48dd-ae97-aea259eb18cf None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:58.569565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20436f96-64b7-48dd-ae97-aea259eb18cf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:58.569900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20436f96-64b7-48dd-ae97-aea259eb18cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:58.589685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-20436f96-64b7-48dd-ae97-aea259eb18cf None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:58.590881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-20436f96-64b7-48dd-ae97-aea259eb18cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:58.649573 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 940/1847] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:58.658383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93283554-e34f-47a4-a208-6ebb4de4251f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:58.658760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93283554-e34f-47a4-a208-6ebb4de4251f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:58.659356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93283554-e34f-47a4-a208-6ebb4de4251f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:58.679753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-93283554-e34f-47a4-a208-6ebb4de4251f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:58.681066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-93283554-e34f-47a4-a208-6ebb4de4251f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:58.689542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 908/1848] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:58.697469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20436f96-64b7-48dd-ae97-aea259eb18cf None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:58.710490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20436f96-64b7-48dd-ae97-aea259eb18cf None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:58.716076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fb336-44a9-4584-9bd7-9173657e3183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:58.716860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fb336-44a9-4584-9bd7-9173657e3183 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:58.717136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fb336-44a9-4584-9bd7-9173657e3183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:58.717551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fb336-44a9-4584-9bd7-9173657e3183 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:58.718192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fb336-44a9-4584-9bd7-9173657e3183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:29:58.718492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fb336-44a9-4584-9bd7-9173657e3183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:58.720258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fb336-44a9-4584-9bd7-9173657e3183 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:58.736579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 941/1849] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:29:58.740612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93283554-e34f-47a4-a208-6ebb4de4251f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:58.751268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93283554-e34f-47a4-a208-6ebb4de4251f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:58.756822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136c64e-4c40-4f3f-8cba-0ba41f1f0d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:58.757541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136c64e-4c40-4f3f-8cba-0ba41f1f0d59 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:58.758063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136c64e-4c40-4f3f-8cba-0ba41f1f0d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:58.758454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136c64e-4c40-4f3f-8cba-0ba41f1f0d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:58.759107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136c64e-4c40-4f3f-8cba-0ba41f1f0d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:58.761049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136c64e-4c40-4f3f-8cba-0ba41f1f0d59 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:58.814444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 909/1850] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:58 2022] POST /identity/v3/projects => generated 407 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:58.816066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fb336-44a9-4584-9bd7-9173657e3183 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:58.826852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fb336-44a9-4584-9bd7-9173657e3183 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:58.900200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308de5f1-5123-4b62-8a07-0d1fc8199657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:58.900868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308de5f1-5123-4b62-8a07-0d1fc8199657 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:58.901147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308de5f1-5123-4b62-8a07-0d1fc8199657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:58.901417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308de5f1-5123-4b62-8a07-0d1fc8199657 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:58.902045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308de5f1-5123-4b62-8a07-0d1fc8199657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:58.903969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308de5f1-5123-4b62-8a07-0d1fc8199657 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:58.927126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 942/1851] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:58 2022] POST /identity/v3/users => generated 304 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:58.931820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136c64e-4c40-4f3f-8cba-0ba41f1f0d59 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:58.943272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136c64e-4c40-4f3f-8cba-0ba41f1f0d59 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:58.948050 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb63c2-a3b3-4a17-858c-487a9f13a36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:58.948683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb63c2-a3b3-4a17-858c-487a9f13a36b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:58.948988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb63c2-a3b3-4a17-858c-487a9f13a36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:58.949260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb63c2-a3b3-4a17-858c-487a9f13a36b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:58.949909 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb63c2-a3b3-4a17-858c-487a9f13a36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:58.951568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb63c2-a3b3-4a17-858c-487a9f13a36b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:58.963411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 910/1852] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:58.967456 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308de5f1-5123-4b62-8a07-0d1fc8199657 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:58.978107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308de5f1-5123-4b62-8a07-0d1fc8199657 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:58.983401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb754b0e-dcfe-4898-a1d0-13331ffd3a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:58.984014 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb754b0e-dcfe-4898-a1d0-13331ffd3a81 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:58.984289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb754b0e-dcfe-4898-a1d0-13331ffd3a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:58.984596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb754b0e-dcfe-4898-a1d0-13331ffd3a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:58.985231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb754b0e-dcfe-4898-a1d0-13331ffd3a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:58.987066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb754b0e-dcfe-4898-a1d0-13331ffd3a81 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:58.998292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 943/1853] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:59.002288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb63c2-a3b3-4a17-858c-487a9f13a36b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:59.013333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb63c2-a3b3-4a17-858c-487a9f13a36b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:59.018237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d0681-698c-48a1-82ea-50c8882a2934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:59.018901 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d0681-698c-48a1-82ea-50c8882a2934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:59.019723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d0681-698c-48a1-82ea-50c8882a2934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:59.019998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d0681-698c-48a1-82ea-50c8882a2934 admin admin] PATH_INFO: `/v3/projects/057763b95c0845d9b578b5e06584bcd6/users/af99f625798e4ba487035d69c7f1da44/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:59.036495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d0681-698c-48a1-82ea-50c8882a2934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=057763b95c0845d9b578b5e06584bcd6, user_id=af99f625798e4ba487035d69c7f1da44, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:59.039291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d0681-698c-48a1-82ea-50c8882a2934 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:59.055843 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 911/1854] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:59 2022] PUT /identity/v3/projects/057763b95c0845d9b578b5e06584bcd6/users/af99f625798e4ba487035d69c7f1da44/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:59.066197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e5b94-88e2-482b-82d9-c8383684e083 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:59.066682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e5b94-88e2-482b-82d9-c8383684e083 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:59.066823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e5b94-88e2-482b-82d9-c8383684e083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:59.095430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-288e5b94-88e2-482b-82d9-c8383684e083 None None] MFA Rules not processed for user `af99f625798e4ba487035d69c7f1da44`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:59.096679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-288e5b94-88e2-482b-82d9-c8383684e083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:59.109979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:59.165691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 944/1855] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:59 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:59.170144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d0681-698c-48a1-82ea-50c8882a2934 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:59.229926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d0681-698c-48a1-82ea-50c8882a2934 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:59.234465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163cd1f-f193-4c57-87dc-400f432a91cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:59.235068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163cd1f-f193-4c57-87dc-400f432a91cf admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:59.235338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163cd1f-f193-4c57-87dc-400f432a91cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:59.235605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163cd1f-f193-4c57-87dc-400f432a91cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:59.236225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163cd1f-f193-4c57-87dc-400f432a91cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:59.238758 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163cd1f-f193-4c57-87dc-400f432a91cf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:59.247676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:29:59.266517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 912/1856] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:29:59 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:59.271178 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e5b94-88e2-482b-82d9-c8383684e083 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:59.281752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e5b94-88e2-482b-82d9-c8383684e083 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:59.337110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9c2f0-88d9-455f-92f6-cedd834921c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:59.337743 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9c2f0-88d9-455f-92f6-cedd834921c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:59.338021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9c2f0-88d9-455f-92f6-cedd834921c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:59.338291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9c2f0-88d9-455f-92f6-cedd834921c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:59.338918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9c2f0-88d9-455f-92f6-cedd834921c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:59.340890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9c2f0-88d9-455f-92f6-cedd834921c9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:59.361504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 945/1857] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:29:59 2022] POST /identity/v3/users => generated 311 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:29:59.365514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163cd1f-f193-4c57-87dc-400f432a91cf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:59.375500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163cd1f-f193-4c57-87dc-400f432a91cf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:59.380623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27651b-633a-4bc8-8b56-3c8fd0f16c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:59.381210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27651b-633a-4bc8-8b56-3c8fd0f16c3c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:59.381485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27651b-633a-4bc8-8b56-3c8fd0f16c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:59.381769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27651b-633a-4bc8-8b56-3c8fd0f16c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:59.382418 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27651b-633a-4bc8-8b56-3c8fd0f16c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:59.384004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27651b-633a-4bc8-8b56-3c8fd0f16c3c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:59.393782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 913/1858] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:59.398629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9c2f0-88d9-455f-92f6-cedd834921c9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:59.408503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9c2f0-88d9-455f-92f6-cedd834921c9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:59.413391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002169af-dbae-498d-9451-8e53dab8a58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:59.414033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002169af-dbae-498d-9451-8e53dab8a58b admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:59.414310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002169af-dbae-498d-9451-8e53dab8a58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:59.414580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002169af-dbae-498d-9451-8e53dab8a58b admin admin] PATH_INFO: `/v3/projects/f0a09c98e1274825873eeafd53b9148b/users/9365c2bd579c4cf7b1805661d5fd78fa/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:59.430861 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002169af-dbae-498d-9451-8e53dab8a58b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a09c98e1274825873eeafd53b9148b, user_id=9365c2bd579c4cf7b1805661d5fd78fa, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:59.433556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002169af-dbae-498d-9451-8e53dab8a58b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:59.448167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 946/1859] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:59 2022] PUT /identity/v3/projects/f0a09c98e1274825873eeafd53b9148b/users/9365c2bd579c4cf7b1805661d5fd78fa/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:59.452189 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27651b-633a-4bc8-8b56-3c8fd0f16c3c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:59.462171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27651b-633a-4bc8-8b56-3c8fd0f16c3c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:59.521478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c29b8-ec6d-490d-b4c1-97d930ad3505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:59.523204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c29b8-ec6d-490d-b4c1-97d930ad3505 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:59.523204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c29b8-ec6d-490d-b4c1-97d930ad3505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:59.523204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c29b8-ec6d-490d-b4c1-97d930ad3505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:59.523545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c29b8-ec6d-490d-b4c1-97d930ad3505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:59.525290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c29b8-ec6d-490d-b4c1-97d930ad3505 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:59.540164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 914/1860] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:29:59.541949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002169af-dbae-498d-9451-8e53dab8a58b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:59.553666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002169af-dbae-498d-9451-8e53dab8a58b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:59.558625 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d553c-d6fd-4ef8-bb97-0f956b0df325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:59.559288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d553c-d6fd-4ef8-bb97-0f956b0df325 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:59.559573 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d553c-d6fd-4ef8-bb97-0f956b0df325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:59.559909 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d553c-d6fd-4ef8-bb97-0f956b0df325 admin admin] PATH_INFO: `/v3/projects/f0a09c98e1274825873eeafd53b9148b/users/9365c2bd579c4cf7b1805661d5fd78fa/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:59.561887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d553c-d6fd-4ef8-bb97-0f956b0df325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a09c98e1274825873eeafd53b9148b, user_id=9365c2bd579c4cf7b1805661d5fd78fa, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:59.564645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d553c-d6fd-4ef8-bb97-0f956b0df325 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:59.590849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 947/1861] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:29:59 2022] PUT /identity/v3/projects/f0a09c98e1274825873eeafd53b9148b/users/9365c2bd579c4cf7b1805661d5fd78fa/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:29:59.601752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e20dfd-996c-479e-88de-50c70185e69d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:59.602039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e20dfd-996c-479e-88de-50c70185e69d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:59.602312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e20dfd-996c-479e-88de-50c70185e69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:59.631742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a5e20dfd-996c-479e-88de-50c70185e69d None None] MFA Rules not processed for user `9365c2bd579c4cf7b1805661d5fd78fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:29:59.632983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a5e20dfd-996c-479e-88de-50c70185e69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:59.708353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 915/1862] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:29:59 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:29:59.718997 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d553c-d6fd-4ef8-bb97-0f956b0df325 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:29:59.720742 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ea9d553c-d6fd-4ef8-bb97-0f956b0df325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:29:59.788002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d553c-d6fd-4ef8-bb97-0f956b0df325 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:29:59.791899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec56f4-b022-4c22-a0fc-67e2bb1bda57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:29:59.792594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec56f4-b022-4c22-a0fc-67e2bb1bda57 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:29:59.792869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec56f4-b022-4c22-a0fc-67e2bb1bda57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:29:59.793136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec56f4-b022-4c22-a0fc-67e2bb1bda57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:29:59.803095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec56f4-b022-4c22-a0fc-67e2bb1bda57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:29:59.804659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec56f4-b022-4c22-a0fc-67e2bb1bda57 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:29:59.815202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 948/1863] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:29:59 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:00.083816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e20dfd-996c-479e-88de-50c70185e69d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:00.097784 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e20dfd-996c-479e-88de-50c70185e69d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:00.099141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90fad6-7956-4031-a1ee-b92b00ad1f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:00.099928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90fad6-7956-4031-a1ee-b92b00ad1f1e service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:00.100307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90fad6-7956-4031-a1ee-b92b00ad1f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:00.100719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90fad6-7956-4031-a1ee-b92b00ad1f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:00.162594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90fad6-7956-4031-a1ee-b92b00ad1f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:00.164108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90fad6-7956-4031-a1ee-b92b00ad1f1e service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:00.168027 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:00.176118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 916/1864] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:00 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:09.810857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:10.176951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:19.260673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec56f4-b022-4c22-a0fc-67e2bb1bda57 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:19.262270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0dec56f4-b022-4c22-a0fc-67e2bb1bda57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:19.329985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec56f4-b022-4c22-a0fc-67e2bb1bda57 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:19.335257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:19.335893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:19.336185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:19.336481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] PATH_INFO: `/v3/users/76f38897be074678a623860affef8a4f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:19.337722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] RBAC: Authorizing `identity:delete_user(user_id=76f38897be074678a623860affef8a4f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:19.340480 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:19.392394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76f38897be074678a623860affef8a4f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:19.404039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76f38897be074678a623860affef8a4f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:19.405324 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76f38897be074678a623860affef8a4f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:19.416614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76f38897be074678a623860affef8a4f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:19.430433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76f38897be074678a623860affef8a4f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:19.439018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 949/1865] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:19 2022] DELETE /identity/v3/users/76f38897be074678a623860affef8a4f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:19.445124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df4d7b-b45e-4791-a936-fa317cbfe59f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:19.445627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df4d7b-b45e-4791-a936-fa317cbfe59f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:19.445897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df4d7b-b45e-4791-a936-fa317cbfe59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:19.476500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-56df4d7b-b45e-4791-a936-fa317cbfe59f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:19.479610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-56df4d7b-b45e-4791-a936-fa317cbfe59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:19.545614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 917/1866] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:19.555322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:19.557336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:19.626622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc845faa-851b-4e36-a85f-82f95c11369d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:19.630825 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409379b7-8686-48ff-be77-e384aeae0979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:19.631435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409379b7-8686-48ff-be77-e384aeae0979 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:19.631702 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409379b7-8686-48ff-be77-e384aeae0979 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:19.631967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409379b7-8686-48ff-be77-e384aeae0979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:19.642304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409379b7-8686-48ff-be77-e384aeae0979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:19.644355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409379b7-8686-48ff-be77-e384aeae0979 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:19.653987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 950/1867] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:20.016585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df4d7b-b45e-4791-a936-fa317cbfe59f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:20.021374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-56df4d7b-b45e-4791-a936-fa317cbfe59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:20.034284 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df4d7b-b45e-4791-a936-fa317cbfe59f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:20.041130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:20.041839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:20.042100 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:20.042368 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] PATH_INFO: `/v3/projects/3054b783f4d04c6ca4f61fa1cc807905` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:20.043488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3054b783f4d04c6ca4f61fa1cc807905)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:20.045463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:20.106219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3054b783f4d04c6ca4f61fa1cc807905', 'request_id': 'req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:20.111928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3054b783f4d04c6ca4f61fa1cc807905', 'request_id': 'req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:20.118982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3054b783f4d04c6ca4f61fa1cc807905', 'request_id': 'req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:20.131471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] The token cache is being invalidate because project 3054b783f4d04c6ca4f61fa1cc807905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:20.131761 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:20.132410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4e812cdd-c1ff-43cb-8004-77a1c7f16c7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:20.133897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 918/1868] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:20 2022] DELETE /identity/v3/projects/3054b783f4d04c6ca4f61fa1cc807905 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:20.138913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b963033-24ce-458a-aa49-9b0d180b5fb6 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:20.139190 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b963033-24ce-458a-aa49-9b0d180b5fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:20.139463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b963033-24ce-458a-aa49-9b0d180b5fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:20.158248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7b963033-24ce-458a-aa49-9b0d180b5fb6 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:20.159908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7b963033-24ce-458a-aa49-9b0d180b5fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:20.177336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:20.224912 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 951/1869] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:20.233950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fe452-78d2-4aed-9a56-4d2eaab400df None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:20.234309 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fe452-78d2-4aed-9a56-4d2eaab400df None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:20.234680 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fe452-78d2-4aed-9a56-4d2eaab400df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:20.256465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cb1fe452-78d2-4aed-9a56-4d2eaab400df None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:20.257705 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cb1fe452-78d2-4aed-9a56-4d2eaab400df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:20.267047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 919/1870] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:20.271686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b963033-24ce-458a-aa49-9b0d180b5fb6 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:20.281321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b963033-24ce-458a-aa49-9b0d180b5fb6 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:20.286647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3fe8a-d97f-495d-ab53-b755f9ee6504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:20.287316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3fe8a-d97f-495d-ab53-b755f9ee6504 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:20.287657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3fe8a-d97f-495d-ab53-b755f9ee6504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:20.288016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3fe8a-d97f-495d-ab53-b755f9ee6504 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:20.288752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3fe8a-d97f-495d-ab53-b755f9ee6504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:30:20.289113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3fe8a-d97f-495d-ab53-b755f9ee6504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:20.290764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3fe8a-d97f-495d-ab53-b755f9ee6504 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:20.303343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 952/1871] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:30:20.307721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fe452-78d2-4aed-9a56-4d2eaab400df None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:20.319832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fe452-78d2-4aed-9a56-4d2eaab400df None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:20.324520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5276f-c5c3-4e3f-b2d0-dd3b63fa2cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:20.325124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5276f-c5c3-4e3f-b2d0-dd3b63fa2cdd admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:20.325396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5276f-c5c3-4e3f-b2d0-dd3b63fa2cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:20.325708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5276f-c5c3-4e3f-b2d0-dd3b63fa2cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:20.326342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5276f-c5c3-4e3f-b2d0-dd3b63fa2cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:20.328610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5276f-c5c3-4e3f-b2d0-dd3b63fa2cdd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:20.358037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 920/1872] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:20 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:20.362042 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3fe8a-d97f-495d-ab53-b755f9ee6504 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:20.372495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3fe8a-d97f-495d-ab53-b755f9ee6504 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:20.435245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f660bfe-925b-4a22-98f2-1b094e08ca47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:20.435860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f660bfe-925b-4a22-98f2-1b094e08ca47 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:20.436137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f660bfe-925b-4a22-98f2-1b094e08ca47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:20.436437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f660bfe-925b-4a22-98f2-1b094e08ca47 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:20.437079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f660bfe-925b-4a22-98f2-1b094e08ca47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:20.439003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f660bfe-925b-4a22-98f2-1b094e08ca47 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:20.459233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 953/1873] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:20 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:20.463434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5276f-c5c3-4e3f-b2d0-dd3b63fa2cdd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:20.474601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5276f-c5c3-4e3f-b2d0-dd3b63fa2cdd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:20.479221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d25fb0-c1ef-4929-9418-9c7915643f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:20.479806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d25fb0-c1ef-4929-9418-9c7915643f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:20.480081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d25fb0-c1ef-4929-9418-9c7915643f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:20.480398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d25fb0-c1ef-4929-9418-9c7915643f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:20.481033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d25fb0-c1ef-4929-9418-9c7915643f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:20.482640 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d25fb0-c1ef-4929-9418-9c7915643f8d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:20.492694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 921/1874] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:20.496311 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f660bfe-925b-4a22-98f2-1b094e08ca47 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:20.505855 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f660bfe-925b-4a22-98f2-1b094e08ca47 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:20.510393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ec0b7-a147-4b1a-8deb-d8ae7519ad89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:20.510973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ec0b7-a147-4b1a-8deb-d8ae7519ad89 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:20.511248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ec0b7-a147-4b1a-8deb-d8ae7519ad89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:20.511518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ec0b7-a147-4b1a-8deb-d8ae7519ad89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:20.512148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ec0b7-a147-4b1a-8deb-d8ae7519ad89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:20.513753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ec0b7-a147-4b1a-8deb-d8ae7519ad89 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:20.525940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 954/1875] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:30:20.529929 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d25fb0-c1ef-4929-9418-9c7915643f8d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:20.541059 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d25fb0-c1ef-4929-9418-9c7915643f8d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:20.546108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b3a42-1c2f-4612-a207-d7ee7a89da0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:20.547037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b3a42-1c2f-4612-a207-d7ee7a89da0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:20.547334 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b3a42-1c2f-4612-a207-d7ee7a89da0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:20.547613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b3a42-1c2f-4612-a207-d7ee7a89da0f admin admin] PATH_INFO: `/v3/projects/4e940e1f7e3140f2865594bcdfc427a0/users/0e76ffd048d240ab94ee7e30b261d3cc/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:20.568176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b3a42-1c2f-4612-a207-d7ee7a89da0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e940e1f7e3140f2865594bcdfc427a0, user_id=0e76ffd048d240ab94ee7e30b261d3cc, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:20.570987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b3a42-1c2f-4612-a207-d7ee7a89da0f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:20.587185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 922/1876] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:20 2022] PUT /identity/v3/projects/4e940e1f7e3140f2865594bcdfc427a0/users/0e76ffd048d240ab94ee7e30b261d3cc/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:20.593283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b298553-11ea-4893-9a48-0ff44272fabb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:20.593579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b298553-11ea-4893-9a48-0ff44272fabb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:20.594794 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b298553-11ea-4893-9a48-0ff44272fabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:20.618572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4b298553-11ea-4893-9a48-0ff44272fabb None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:20.620141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4b298553-11ea-4893-9a48-0ff44272fabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:20.778994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 955/1877] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:20.835929 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b3a42-1c2f-4612-a207-d7ee7a89da0f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:20.838730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-db4b3a42-1c2f-4612-a207-d7ee7a89da0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:20.907526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:21.094587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b3a42-1c2f-4612-a207-d7ee7a89da0f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:21.098940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f1b5e-1795-4d49-9fba-5c4ee8f41fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:21.099633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1b5e-1795-4d49-9fba-5c4ee8f41fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:21.099997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1b5e-1795-4d49-9fba-5c4ee8f41fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:21.100721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1b5e-1795-4d49-9fba-5c4ee8f41fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:21.126176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f1b5e-1795-4d49-9fba-5c4ee8f41fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:21.130459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f1b5e-1795-4d49-9fba-5c4ee8f41fb5 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:21.155834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 923/1878] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:22.726622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b298553-11ea-4893-9a48-0ff44272fabb None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:22.728766 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4b298553-11ea-4893-9a48-0ff44272fabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:22.744035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b298553-11ea-4893-9a48-0ff44272fabb None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:22.753399 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:22.755722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:22.756271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:22.756801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] PATH_INFO: `/v3/users/af99f625798e4ba487035d69c7f1da44` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:22.762628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af99f625798e4ba487035d69c7f1da44)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:22.765257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:22.831505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:22.860958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af99f625798e4ba487035d69c7f1da44'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:22.877310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af99f625798e4ba487035d69c7f1da44'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:22.878142 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af99f625798e4ba487035d69c7f1da44'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:22.897228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af99f625798e4ba487035d69c7f1da44'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:22.923430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af99f625798e4ba487035d69c7f1da44'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:22.940927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 956/1879] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:22 2022] DELETE /identity/v3/users/af99f625798e4ba487035d69c7f1da44 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:22.945876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90db63-f979-4c4d-a51d-63604de49fe4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:22.946424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90db63-f979-4c4d-a51d-63604de49fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:22.946424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90db63-f979-4c4d-a51d-63604de49fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:22.973754 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8c90db63-f979-4c4d-a51d-63604de49fe4 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:22.975411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8c90db63-f979-4c4d-a51d-63604de49fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:23.045791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 924/1880] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:23.056630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:23.058928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:23.123679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12e910-6f57-4739-8ff5-c1fd7b9529f6 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:23.128547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffa918-3eb2-4d87-8ae0-8f8e22f0e75e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:23.128920 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffa918-3eb2-4d87-8ae0-8f8e22f0e75e service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:23.129241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffa918-3eb2-4d87-8ae0-8f8e22f0e75e service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:23.129511 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffa918-3eb2-4d87-8ae0-8f8e22f0e75e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:23.146410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffa918-3eb2-4d87-8ae0-8f8e22f0e75e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:23.148019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffa918-3eb2-4d87-8ae0-8f8e22f0e75e service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:23.172562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 957/1881] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:30:23.845018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90db63-f979-4c4d-a51d-63604de49fe4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:23.847166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8c90db63-f979-4c4d-a51d-63604de49fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:23.860934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90db63-f979-4c4d-a51d-63604de49fe4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:23.866120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:23.867161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:23.867587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:23.874327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] PATH_INFO: `/v3/projects/057763b95c0845d9b578b5e06584bcd6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:23.875879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=057763b95c0845d9b578b5e06584bcd6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:23.879155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:23.977160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057763b95c0845d9b578b5e06584bcd6', 'request_id': 'req-525c9755-1391-4313-b74c-b46c0850f7fe'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:23.984001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057763b95c0845d9b578b5e06584bcd6', 'request_id': 'req-525c9755-1391-4313-b74c-b46c0850f7fe'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:23.993129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057763b95c0845d9b578b5e06584bcd6', 'request_id': 'req-525c9755-1391-4313-b74c-b46c0850f7fe'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:24.014765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] The token cache is being invalidate because project 057763b95c0845d9b578b5e06584bcd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:24.015688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:24.016106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:24.019477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 925/1882] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:23 2022] DELETE /identity/v3/projects/057763b95c0845d9b578b5e06584bcd6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:24.024588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffa918-3eb2-4d87-8ae0-8f8e22f0e75e service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:24.040006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffa918-3eb2-4d87-8ae0-8f8e22f0e75e service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:24.125087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03b1f6-5fbd-40cd-b7ed-f1f8f91a85c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:24.125747 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b1f6-5fbd-40cd-b7ed-f1f8f91a85c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:24.126021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b1f6-5fbd-40cd-b7ed-f1f8f91a85c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:24.126290 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b1f6-5fbd-40cd-b7ed-f1f8f91a85c6 admin admin] PATH_INFO: `/v3/users/9365c2bd579c4cf7b1805661d5fd78fa` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:24.127459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03b1f6-5fbd-40cd-b7ed-f1f8f91a85c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9365c2bd579c4cf7b1805661d5fd78fa)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:24.129976 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03b1f6-5fbd-40cd-b7ed-f1f8f91a85c6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:24.193691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:24.195096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4f03b1f6-5fbd-40cd-b7ed-f1f8f91a85c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9365c2bd579c4cf7b1805661d5fd78fa'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:24.209581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4f03b1f6-5fbd-40cd-b7ed-f1f8f91a85c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9365c2bd579c4cf7b1805661d5fd78fa'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:24.210576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4f03b1f6-5fbd-40cd-b7ed-f1f8f91a85c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9365c2bd579c4cf7b1805661d5fd78fa'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:24.221407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4f03b1f6-5fbd-40cd-b7ed-f1f8f91a85c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9365c2bd579c4cf7b1805661d5fd78fa'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:24.244256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4f03b1f6-5fbd-40cd-b7ed-f1f8f91a85c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9365c2bd579c4cf7b1805661d5fd78fa'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:24.252429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 958/1883] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:24 2022] DELETE /identity/v3/users/9365c2bd579c4cf7b1805661d5fd78fa => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:24.848333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:24.850111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:24.881070 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c9755-1391-4313-b74c-b46c0850f7fe admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:24.978473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:25.000845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:25.001857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:25.002312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:25.003367 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] PATH_INFO: `/v3/projects/f0a09c98e1274825873eeafd53b9148b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:25.004888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0a09c98e1274825873eeafd53b9148b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:25.007465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:25.130561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0a09c98e1274825873eeafd53b9148b', 'request_id': 'req-72b28cae-2322-4839-af60-54ff871dab5f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:25.155772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0a09c98e1274825873eeafd53b9148b', 'request_id': 'req-72b28cae-2322-4839-af60-54ff871dab5f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:25.163535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0a09c98e1274825873eeafd53b9148b', 'request_id': 'req-72b28cae-2322-4839-af60-54ff871dab5f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:25.189265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] The token cache is being invalidate because project f0a09c98e1274825873eeafd53b9148b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:25.190952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:25.191823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-72b28cae-2322-4839-af60-54ff871dab5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:25.194358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 926/1884] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:24 2022] DELETE /identity/v3/projects/f0a09c98e1274825873eeafd53b9148b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:25.198987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf34ff-483f-4919-9e28-a3bf92828cb7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:25.199458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf34ff-483f-4919-9e28-a3bf92828cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:25.199534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf34ff-483f-4919-9e28-a3bf92828cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:25.224588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-96cf34ff-483f-4919-9e28-a3bf92828cb7 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:25.227618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-96cf34ff-483f-4919-9e28-a3bf92828cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:25.393944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 959/1885] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:25.401445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6fc96-09c5-4ac7-974a-37b268f04477 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:25.401727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6fc96-09c5-4ac7-974a-37b268f04477 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:25.402004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6fc96-09c5-4ac7-974a-37b268f04477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:25.456324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-73f6fc96-09c5-4ac7-974a-37b268f04477 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:25.457712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-73f6fc96-09c5-4ac7-974a-37b268f04477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:25.492779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 927/1886] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:25.496975 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf34ff-483f-4919-9e28-a3bf92828cb7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:25.510573 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf34ff-483f-4919-9e28-a3bf92828cb7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:25.520540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b677e3d-48c8-4598-8b24-26f9ae73bfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:25.521179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b677e3d-48c8-4598-8b24-26f9ae73bfbc admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:25.521454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b677e3d-48c8-4598-8b24-26f9ae73bfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:25.521720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b677e3d-48c8-4598-8b24-26f9ae73bfbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:25.522393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b677e3d-48c8-4598-8b24-26f9ae73bfbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:30:25.522670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b677e3d-48c8-4598-8b24-26f9ae73bfbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:25.524244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b677e3d-48c8-4598-8b24-26f9ae73bfbc admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:25.553548 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 960/1887] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:30:25.557946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6fc96-09c5-4ac7-974a-37b268f04477 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:25.568741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6fc96-09c5-4ac7-974a-37b268f04477 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:25.574681 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c7460-1d58-43f8-9aa6-d82c68134e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:25.575361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c7460-1d58-43f8-9aa6-d82c68134e9f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:25.575667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c7460-1d58-43f8-9aa6-d82c68134e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:25.575991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c7460-1d58-43f8-9aa6-d82c68134e9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:25.576762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c7460-1d58-43f8-9aa6-d82c68134e9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:25.578686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c7460-1d58-43f8-9aa6-d82c68134e9f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:25.621009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 928/1888] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:25 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:25.625577 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b677e3d-48c8-4598-8b24-26f9ae73bfbc admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:25.636288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b677e3d-48c8-4598-8b24-26f9ae73bfbc admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:25.710517 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9c5fa-c9fe-4a4e-a9a2-89ab071d6788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:25.711141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9c5fa-c9fe-4a4e-a9a2-89ab071d6788 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:25.711419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9c5fa-c9fe-4a4e-a9a2-89ab071d6788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:25.711691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9c5fa-c9fe-4a4e-a9a2-89ab071d6788 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:25.712323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9c5fa-c9fe-4a4e-a9a2-89ab071d6788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:25.714280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9c5fa-c9fe-4a4e-a9a2-89ab071d6788 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:25.745654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 961/1889] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:25 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:25.749807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c7460-1d58-43f8-9aa6-d82c68134e9f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:25.761892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c7460-1d58-43f8-9aa6-d82c68134e9f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:25.770149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13c923-fce7-4780-9c5b-0c7d7f411fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:25.770873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13c923-fce7-4780-9c5b-0c7d7f411fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:25.771230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13c923-fce7-4780-9c5b-0c7d7f411fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:25.771579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13c923-fce7-4780-9c5b-0c7d7f411fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:25.772321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13c923-fce7-4780-9c5b-0c7d7f411fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:25.774003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13c923-fce7-4780-9c5b-0c7d7f411fe5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:25.786788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 929/1890] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:30:25.790848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9c5fa-c9fe-4a4e-a9a2-89ab071d6788 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:25.801927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9c5fa-c9fe-4a4e-a9a2-89ab071d6788 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:25.810134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445be0a-b3e4-4950-8699-62958fc5ab46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:25.811229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445be0a-b3e4-4950-8699-62958fc5ab46 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:25.812283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445be0a-b3e4-4950-8699-62958fc5ab46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:25.812683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445be0a-b3e4-4950-8699-62958fc5ab46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:25.814859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445be0a-b3e4-4950-8699-62958fc5ab46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:25.818059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445be0a-b3e4-4950-8699-62958fc5ab46 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:25.833109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 962/1891] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:25.837200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13c923-fce7-4780-9c5b-0c7d7f411fe5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:25.849222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13c923-fce7-4780-9c5b-0c7d7f411fe5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:25.854090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2538cc-b05f-4990-9a6e-7b2b95f9fde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:25.854770 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2538cc-b05f-4990-9a6e-7b2b95f9fde2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:25.855048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2538cc-b05f-4990-9a6e-7b2b95f9fde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:25.855464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2538cc-b05f-4990-9a6e-7b2b95f9fde2 admin admin] PATH_INFO: `/v3/projects/1423d6f63a894abf97d6e8eeb97b259f/users/56d90efe060a42678a0a4794013a50ab/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:25.875357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2538cc-b05f-4990-9a6e-7b2b95f9fde2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1423d6f63a894abf97d6e8eeb97b259f, user_id=56d90efe060a42678a0a4794013a50ab, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:25.878063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2538cc-b05f-4990-9a6e-7b2b95f9fde2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:25.901156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 930/1892] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:25 2022] PUT /identity/v3/projects/1423d6f63a894abf97d6e8eeb97b259f/users/56d90efe060a42678a0a4794013a50ab/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:25.912058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40045548-14b2-4d8a-91bb-6634a977304b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:25.912614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40045548-14b2-4d8a-91bb-6634a977304b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:25.913046 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40045548-14b2-4d8a-91bb-6634a977304b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:25.960200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-40045548-14b2-4d8a-91bb-6634a977304b None None] MFA Rules not processed for user `56d90efe060a42678a0a4794013a50ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:25.961524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-40045548-14b2-4d8a-91bb-6634a977304b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:25.988622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:26.086199 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 963/1893] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:26.090559 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2538cc-b05f-4990-9a6e-7b2b95f9fde2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:26.182441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2538cc-b05f-4990-9a6e-7b2b95f9fde2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:26.189218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224c5a1-1e92-4356-a147-37546a5b5ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:26.190174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224c5a1-1e92-4356-a147-37546a5b5ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:26.190519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224c5a1-1e92-4356-a147-37546a5b5ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:26.190966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224c5a1-1e92-4356-a147-37546a5b5ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:26.191604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224c5a1-1e92-4356-a147-37546a5b5ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:26.194477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224c5a1-1e92-4356-a147-37546a5b5ca7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:26.228241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:26.247840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 931/1894] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:26 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:30:26.251138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40045548-14b2-4d8a-91bb-6634a977304b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:26.270461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40045548-14b2-4d8a-91bb-6634a977304b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:26.356871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddd26d-0978-47d9-9840-62db4284e682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:26.358295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddd26d-0978-47d9-9840-62db4284e682 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:26.358295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddd26d-0978-47d9-9840-62db4284e682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:26.359534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddd26d-0978-47d9-9840-62db4284e682 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:26.360310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dddd26d-0978-47d9-9840-62db4284e682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:26.366041 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dddd26d-0978-47d9-9840-62db4284e682 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:26.431509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 964/1895] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:26 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:30:26.433326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224c5a1-1e92-4356-a147-37546a5b5ca7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:26.447532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224c5a1-1e92-4356-a147-37546a5b5ca7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:26.452562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e819a6b-0ee4-45b8-a700-608f2606652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:26.453176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e819a6b-0ee4-45b8-a700-608f2606652e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:26.453457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e819a6b-0ee4-45b8-a700-608f2606652e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:26.453724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e819a6b-0ee4-45b8-a700-608f2606652e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:26.454380 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e819a6b-0ee4-45b8-a700-608f2606652e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:26.457399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e819a6b-0ee4-45b8-a700-608f2606652e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:26.475484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 932/1896] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:26.475648 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddd26d-0978-47d9-9840-62db4284e682 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:26.489464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddd26d-0978-47d9-9840-62db4284e682 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:26.494605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd99af7-aef7-447a-9f04-00b5ba28a6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:26.495357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd99af7-aef7-447a-9f04-00b5ba28a6fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:26.495706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd99af7-aef7-447a-9f04-00b5ba28a6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:26.496056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd99af7-aef7-447a-9f04-00b5ba28a6fb admin admin] PATH_INFO: `/v3/projects/b4b1eb30c96a4d7196993641d8e94074/users/c277638d965542eebcde4e45aca265a3/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:26.514670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd99af7-aef7-447a-9f04-00b5ba28a6fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b1eb30c96a4d7196993641d8e94074, user_id=c277638d965542eebcde4e45aca265a3, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:26.517685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd99af7-aef7-447a-9f04-00b5ba28a6fb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:26.543279 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 965/1897] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:26 2022] PUT /identity/v3/projects/b4b1eb30c96a4d7196993641d8e94074/users/c277638d965542eebcde4e45aca265a3/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:26.544642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e819a6b-0ee4-45b8-a700-608f2606652e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:26.574263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e819a6b-0ee4-45b8-a700-608f2606652e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:26.648735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d58b43-4016-4f1d-b993-2147fbc46b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:26.649468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d58b43-4016-4f1d-b993-2147fbc46b0e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:26.649733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d58b43-4016-4f1d-b993-2147fbc46b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:26.650083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d58b43-4016-4f1d-b993-2147fbc46b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:26.651333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d58b43-4016-4f1d-b993-2147fbc46b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:26.654907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d58b43-4016-4f1d-b993-2147fbc46b0e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:26.670919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 933/1898] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:26.672474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd99af7-aef7-447a-9f04-00b5ba28a6fb admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:26.685411 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd99af7-aef7-447a-9f04-00b5ba28a6fb admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:26.695601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81b9c0-457b-406c-ad1e-678409f6be62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:26.696454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81b9c0-457b-406c-ad1e-678409f6be62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:26.696805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81b9c0-457b-406c-ad1e-678409f6be62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:26.697608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81b9c0-457b-406c-ad1e-678409f6be62 admin admin] PATH_INFO: `/v3/projects/b4b1eb30c96a4d7196993641d8e94074/users/c277638d965542eebcde4e45aca265a3/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:26.700000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81b9c0-457b-406c-ad1e-678409f6be62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b1eb30c96a4d7196993641d8e94074, user_id=c277638d965542eebcde4e45aca265a3, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:26.704840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81b9c0-457b-406c-ad1e-678409f6be62 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:26.724466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 966/1899] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:26 2022] PUT /identity/v3/projects/b4b1eb30c96a4d7196993641d8e94074/users/c277638d965542eebcde4e45aca265a3/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:26.732569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1dbc-046a-4e9f-b759-65d28b3e4560 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:26.732857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1dbc-046a-4e9f-b759-65d28b3e4560 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:26.733129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1dbc-046a-4e9f-b759-65d28b3e4560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:26.778266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d71d1dbc-046a-4e9f-b759-65d28b3e4560 None None] MFA Rules not processed for user `c277638d965542eebcde4e45aca265a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:26.779874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d71d1dbc-046a-4e9f-b759-65d28b3e4560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:26.877179 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 934/1900] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:26.887943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81b9c0-457b-406c-ad1e-678409f6be62 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:26.889778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bc81b9c0-457b-406c-ad1e-678409f6be62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:26.976594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81b9c0-457b-406c-ad1e-678409f6be62 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:26.980800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed01b1-baf6-4d89-a1ec-8eb8187afeef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:26.981455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed01b1-baf6-4d89-a1ec-8eb8187afeef service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:26.981736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed01b1-baf6-4d89-a1ec-8eb8187afeef service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:26.982013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed01b1-baf6-4d89-a1ec-8eb8187afeef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:27.018174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:27.119250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed01b1-baf6-4d89-a1ec-8eb8187afeef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:27.121254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed01b1-baf6-4d89-a1ec-8eb8187afeef service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:27.139974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 967/1901] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:27.767562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b22b8-621e-4379-9d5f-4175865d62d0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:27.768002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b22b8-621e-4379-9d5f-4175865d62d0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:27.768130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b22b8-621e-4379-9d5f-4175865d62d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:27.821831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7f0b22b8-621e-4379-9d5f-4175865d62d0 None None] MFA Rules not processed for user `0e76ffd048d240ab94ee7e30b261d3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:27.823340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7f0b22b8-621e-4379-9d5f-4175865d62d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:27.903808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:27.923712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 935/1902] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:27.927986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed01b1-baf6-4d89-a1ec-8eb8187afeef service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:27.929712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-03ed01b1-baf6-4d89-a1ec-8eb8187afeef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:28.028289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed01b1-baf6-4d89-a1ec-8eb8187afeef service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:28.033366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbf3e7-3cac-4e04-9794-4ff09996aeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:28.034318 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbf3e7-3cac-4e04-9794-4ff09996aeef admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:28.034622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbf3e7-3cac-4e04-9794-4ff09996aeef admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:28.034854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbf3e7-3cac-4e04-9794-4ff09996aeef admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:28.035501 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbf3e7-3cac-4e04-9794-4ff09996aeef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:28.037460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbf3e7-3cac-4e04-9794-4ff09996aeef admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:28.081448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 968/1903] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:27 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:30:28.082657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b22b8-621e-4379-9d5f-4175865d62d0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:28.094250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b22b8-621e-4379-9d5f-4175865d62d0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:28.155489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fbe31-cfbe-4faa-8403-a22dbda8e458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:28.156133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fbe31-cfbe-4faa-8403-a22dbda8e458 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:28.156387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fbe31-cfbe-4faa-8403-a22dbda8e458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:28.156735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fbe31-cfbe-4faa-8403-a22dbda8e458 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:28.157364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fbe31-cfbe-4faa-8403-a22dbda8e458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:28.160998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fbe31-cfbe-4faa-8403-a22dbda8e458 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:28.190186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 936/1904] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:28 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:28.190792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbf3e7-3cac-4e04-9794-4ff09996aeef admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:28.211413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbf3e7-3cac-4e04-9794-4ff09996aeef admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:28.217486 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2116e-7dd2-4318-8681-e93439db932a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:28.217943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2116e-7dd2-4318-8681-e93439db932a admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:28.218221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2116e-7dd2-4318-8681-e93439db932a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:28.218488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2116e-7dd2-4318-8681-e93439db932a admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:28.219127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2116e-7dd2-4318-8681-e93439db932a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:28.221011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2116e-7dd2-4318-8681-e93439db932a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:28.236442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 969/1905] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:30:28.238800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fbe31-cfbe-4faa-8403-a22dbda8e458 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:28.249166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fbe31-cfbe-4faa-8403-a22dbda8e458 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:28.254229 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef0dbc-cae0-40bc-b4ae-6e81b90cb3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:28.255174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0dbc-cae0-40bc-b4ae-6e81b90cb3bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:28.255555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0dbc-cae0-40bc-b4ae-6e81b90cb3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:28.255924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0dbc-cae0-40bc-b4ae-6e81b90cb3bd admin admin] PATH_INFO: `/v3/projects/a780a162c99a4b82add018c78e7d4eb8/users/21645f5c03d6474bb1fd5e8750b6b500/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:28.276166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef0dbc-cae0-40bc-b4ae-6e81b90cb3bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a780a162c99a4b82add018c78e7d4eb8, user_id=21645f5c03d6474bb1fd5e8750b6b500, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:28.279582 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef0dbc-cae0-40bc-b4ae-6e81b90cb3bd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:28.305358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 937/1906] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:28 2022] PUT /identity/v3/projects/a780a162c99a4b82add018c78e7d4eb8/users/21645f5c03d6474bb1fd5e8750b6b500/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:28.309578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2116e-7dd2-4318-8681-e93439db932a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:28.325166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2116e-7dd2-4318-8681-e93439db932a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:28.412032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702364c-d761-495a-84d6-f985d6a1d46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:28.412698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702364c-d761-495a-84d6-f985d6a1d46b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:28.412970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702364c-d761-495a-84d6-f985d6a1d46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:28.413236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702364c-d761-495a-84d6-f985d6a1d46b admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:28.413876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702364c-d761-495a-84d6-f985d6a1d46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:28.415453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702364c-d761-495a-84d6-f985d6a1d46b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:28.432107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 970/1907] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:28.433968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef0dbc-cae0-40bc-b4ae-6e81b90cb3bd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:28.444880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef0dbc-cae0-40bc-b4ae-6e81b90cb3bd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:28.449508 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7167b6-6945-496d-88d1-e0eed2196bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:28.450098 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7167b6-6945-496d-88d1-e0eed2196bee admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:28.450391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7167b6-6945-496d-88d1-e0eed2196bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:28.450678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7167b6-6945-496d-88d1-e0eed2196bee admin admin] PATH_INFO: `/v3/projects/a780a162c99a4b82add018c78e7d4eb8/users/21645f5c03d6474bb1fd5e8750b6b500/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:28.453006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7167b6-6945-496d-88d1-e0eed2196bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=a780a162c99a4b82add018c78e7d4eb8, user_id=21645f5c03d6474bb1fd5e8750b6b500, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:28.455882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7167b6-6945-496d-88d1-e0eed2196bee admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:28.472589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 938/1908] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:28 2022] PUT /identity/v3/projects/a780a162c99a4b82add018c78e7d4eb8/users/21645f5c03d6474bb1fd5e8750b6b500/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:33.822477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702364c-d761-495a-84d6-f985d6a1d46b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:33.824485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e702364c-d761-495a-84d6-f985d6a1d46b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:33.854711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702364c-d761-495a-84d6-f985d6a1d46b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:33.940589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:34.044239 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd78fe-d3a4-4007-ad57-52a10c81d208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:34.047834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd78fe-d3a4-4007-ad57-52a10c81d208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:34.048869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd78fe-d3a4-4007-ad57-52a10c81d208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:34.049720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd78fe-d3a4-4007-ad57-52a10c81d208 admin admin] PATH_INFO: `/v3/users/8b1bf69b827443e79921128845a70566` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:34.056066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd78fe-d3a4-4007-ad57-52a10c81d208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1bf69b827443e79921128845a70566)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:34.058170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd78fe-d3a4-4007-ad57-52a10c81d208 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:34.240203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-71bd78fe-d3a4-4007-ad57-52a10c81d208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1bf69b827443e79921128845a70566'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:34.249196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-71bd78fe-d3a4-4007-ad57-52a10c81d208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1bf69b827443e79921128845a70566'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:34.250272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-71bd78fe-d3a4-4007-ad57-52a10c81d208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1bf69b827443e79921128845a70566'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:34.261265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-71bd78fe-d3a4-4007-ad57-52a10c81d208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1bf69b827443e79921128845a70566'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:34.271032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7167b6-6945-496d-88d1-e0eed2196bee admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:34.272122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b7167b6-6945-496d-88d1-e0eed2196bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:34.314612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-71bd78fe-d3a4-4007-ad57-52a10c81d208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1bf69b827443e79921128845a70566'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:34.336781 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 971/1909] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:33 2022] DELETE /identity/v3/users/8b1bf69b827443e79921128845a70566 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:34.338887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e9b3d-1eec-4a4c-99d8-f4321521abfa None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:34.339326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e9b3d-1eec-4a4c-99d8-f4321521abfa None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:34.341244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e9b3d-1eec-4a4c-99d8-f4321521abfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:34.375395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f37e9b3d-1eec-4a4c-99d8-f4321521abfa None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:34.379409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f37e9b3d-1eec-4a4c-99d8-f4321521abfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:34.505507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7167b6-6945-496d-88d1-e0eed2196bee admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:34.542564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 972/1910] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:30:34.600057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e9b3d-1eec-4a4c-99d8-f4321521abfa None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:34.601674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f37e9b3d-1eec-4a4c-99d8-f4321521abfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:34.614841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33683da-45aa-4ae3-a585-419f56d827e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:34.615414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33683da-45aa-4ae3-a585-419f56d827e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:34.615678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33683da-45aa-4ae3-a585-419f56d827e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:34.616426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33683da-45aa-4ae3-a585-419f56d827e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:34.632968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e33683da-45aa-4ae3-a585-419f56d827e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:34.729677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:34.818326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e9b3d-1eec-4a4c-99d8-f4321521abfa None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:34.830048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e09b54-e6a1-452b-8140-3b73de606e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:34.831193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e09b54-e6a1-452b-8140-3b73de606e65 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:34.831463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e09b54-e6a1-452b-8140-3b73de606e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:34.831726 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e09b54-e6a1-452b-8140-3b73de606e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:34.865841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e09b54-e6a1-452b-8140-3b73de606e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:34.870360 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e09b54-e6a1-452b-8140-3b73de606e65 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:34.874674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:34.891230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33683da-45aa-4ae3-a585-419f56d827e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:34.892545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33683da-45aa-4ae3-a585-419f56d827e3 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:34.907294 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 973/1911] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:34.938599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 939/1912] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:36.856707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e09b54-e6a1-452b-8140-3b73de606e65 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:36.858727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-56e09b54-e6a1-452b-8140-3b73de606e65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:36.892546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e09b54-e6a1-452b-8140-3b73de606e65 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:36.904132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:36.905141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:36.905667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:36.907251 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] PATH_INFO: `/v3/projects/2c308897e7854e3a8b3a7d7a65065b0b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:36.909728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c308897e7854e3a8b3a7d7a65065b0b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:36.915587 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:37.093949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c308897e7854e3a8b3a7d7a65065b0b', 'request_id': 'req-87d8e344-d0fd-4e99-a492-730929f725ec'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:37.118231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c308897e7854e3a8b3a7d7a65065b0b', 'request_id': 'req-87d8e344-d0fd-4e99-a492-730929f725ec'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:37.166430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c308897e7854e3a8b3a7d7a65065b0b', 'request_id': 'req-87d8e344-d0fd-4e99-a492-730929f725ec'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:37.215997 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] The token cache is being invalidate because project 2c308897e7854e3a8b3a7d7a65065b0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:37.218850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:37.219816 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:37.227248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 974/1913] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:36 2022] DELETE /identity/v3/projects/2c308897e7854e3a8b3a7d7a65065b0b => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:37.232455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33683da-45aa-4ae3-a585-419f56d827e3 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:37.264765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33683da-45aa-4ae3-a585-419f56d827e3 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:37.401717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d075426-bd21-431d-bf02-a34b794da74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:37.401941 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d075426-bd21-431d-bf02-a34b794da74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:37.402194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d075426-bd21-431d-bf02-a34b794da74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:37.402467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d075426-bd21-431d-bf02-a34b794da74e admin admin] PATH_INFO: `/v3/users/d5dbd31355cf44d5a88f4a8b6d3fe074` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:37.406489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d075426-bd21-431d-bf02-a34b794da74e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5dbd31355cf44d5a88f4a8b6d3fe074)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:37.409536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d075426-bd21-431d-bf02-a34b794da74e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:37.536975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3d075426-bd21-431d-bf02-a34b794da74e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5dbd31355cf44d5a88f4a8b6d3fe074'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:37.561886 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3d075426-bd21-431d-bf02-a34b794da74e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5dbd31355cf44d5a88f4a8b6d3fe074'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:37.564435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3d075426-bd21-431d-bf02-a34b794da74e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5dbd31355cf44d5a88f4a8b6d3fe074'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:37.579569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3d075426-bd21-431d-bf02-a34b794da74e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5dbd31355cf44d5a88f4a8b6d3fe074'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:37.611106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3d075426-bd21-431d-bf02-a34b794da74e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5dbd31355cf44d5a88f4a8b6d3fe074'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:37.634014 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 940/1914] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:37 2022] DELETE /identity/v3/users/d5dbd31355cf44d5a88f4a8b6d3fe074 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:38.622191 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:38.624222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:38.650226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8e344-d0fd-4e99-a492-730929f725ec admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:38.779475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:38.780419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:38.784987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:38.785701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] PATH_INFO: `/v3/projects/2c9dd2b1d1df4060b9f1977880fa491b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:38.787217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c9dd2b1d1df4060b9f1977880fa491b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:38.793956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:38.987274 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c9dd2b1d1df4060b9f1977880fa491b', 'request_id': 'req-742156b5-a012-4a15-882e-17e8c89c8328'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:38.999459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c9dd2b1d1df4060b9f1977880fa491b', 'request_id': 'req-742156b5-a012-4a15-882e-17e8c89c8328'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:39.022330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c9dd2b1d1df4060b9f1977880fa491b', 'request_id': 'req-742156b5-a012-4a15-882e-17e8c89c8328'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:39.057916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] The token cache is being invalidate because project 2c9dd2b1d1df4060b9f1977880fa491b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:39.058386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:39.059224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-742156b5-a012-4a15-882e-17e8c89c8328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:39.061711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 975/1915] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:38 2022] DELETE /identity/v3/projects/2c9dd2b1d1df4060b9f1977880fa491b => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:39.074859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c731789-836c-410f-b61e-2f7cc366b1ff None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:39.089586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c731789-836c-410f-b61e-2f7cc366b1ff None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:39.100291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c731789-836c-410f-b61e-2f7cc366b1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:39.136545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d400e-67b6-4c4c-ae2c-628b17178cc2 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:39.137344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d400e-67b6-4c4c-ae2c-628b17178cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:39.138303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d400e-67b6-4c4c-ae2c-628b17178cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:39.149562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2c731789-836c-410f-b61e-2f7cc366b1ff None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:39.152439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2c731789-836c-410f-b61e-2f7cc366b1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:39.192358 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:39.228057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1a6d400e-67b6-4c4c-ae2c-628b17178cc2 None None] MFA Rules not processed for user `21645f5c03d6474bb1fd5e8750b6b500`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:39.228897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:39.230567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1a6d400e-67b6-4c4c-ae2c-628b17178cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:39.304612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 941/1916] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:39.312046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff25bb-1284-44a3-96ec-9e21754a9fda None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:39.312352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff25bb-1284-44a3-96ec-9e21754a9fda None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:39.312694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff25bb-1284-44a3-96ec-9e21754a9fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:39.373469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c5ff25bb-1284-44a3-96ec-9e21754a9fda None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:39.375266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c5ff25bb-1284-44a3-96ec-9e21754a9fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:39.416247 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 942/1917] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:30:39.421410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff25bb-1284-44a3-96ec-9e21754a9fda None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:39.443243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 976/1918] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:39.447391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff25bb-1284-44a3-96ec-9e21754a9fda None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:39.458809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2d6d8-8ab2-47bd-a971-1a275e783d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:39.459797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2d6d8-8ab2-47bd-a971-1a275e783d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:39.460671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2d6d8-8ab2-47bd-a971-1a275e783d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:39.461435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2d6d8-8ab2-47bd-a971-1a275e783d6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:39.462322 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2d6d8-8ab2-47bd-a971-1a275e783d6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:30:39.462510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d400e-67b6-4c4c-ae2c-628b17178cc2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:39.463440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2d6d8-8ab2-47bd-a971-1a275e783d6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:39.464118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1a6d400e-67b6-4c4c-ae2c-628b17178cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:39.468299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2d6d8-8ab2-47bd-a971-1a275e783d6a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:39.532216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 943/1919] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:30:39.535187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2d6d8-8ab2-47bd-a971-1a275e783d6a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:39.558958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2d6d8-8ab2-47bd-a971-1a275e783d6a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:39.573867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce4c66-9f73-46c6-9560-d279e43da6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:39.574500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce4c66-9f73-46c6-9560-d279e43da6ca admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:39.574779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce4c66-9f73-46c6-9560-d279e43da6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:39.575049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce4c66-9f73-46c6-9560-d279e43da6ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:39.575689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce4c66-9f73-46c6-9560-d279e43da6ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:39.579365 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce4c66-9f73-46c6-9560-d279e43da6ca admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:39.690619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 944/1920] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:39 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:39.691818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce4c66-9f73-46c6-9560-d279e43da6ca admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:39.733091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d400e-67b6-4c4c-ae2c-628b17178cc2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:39.864803 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce4c66-9f73-46c6-9560-d279e43da6ca admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:39.882126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ee7d9-ad27-454d-8eb6-995dde07cb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:39.882768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ee7d9-ad27-454d-8eb6-995dde07cb0c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:39.883048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ee7d9-ad27-454d-8eb6-995dde07cb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:39.888465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ee7d9-ad27-454d-8eb6-995dde07cb0c admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:39.902874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ee7d9-ad27-454d-8eb6-995dde07cb0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:39.915525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ee7d9-ad27-454d-8eb6-995dde07cb0c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:39.953713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81f611-8143-44c6-b8a4-3e2cacd604bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:39.953713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81f611-8143-44c6-b8a4-3e2cacd604bb service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:39.953713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81f611-8143-44c6-b8a4-3e2cacd604bb service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:39.954060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81f611-8143-44c6-b8a4-3e2cacd604bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:39.958907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fb81f611-8143-44c6-b8a4-3e2cacd604bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:39.992295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 945/1921] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:39 2022] POST /identity/v3/users => generated 284 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:39.994456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ee7d9-ad27-454d-8eb6-995dde07cb0c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:40.024026 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ee7d9-ad27-454d-8eb6-995dde07cb0c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:40.054544 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:40.083627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1558bb8-e71c-4f26-a3c7-de02efb41bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:40.084280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1558bb8-e71c-4f26-a3c7-de02efb41bfb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:40.086527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1558bb8-e71c-4f26-a3c7-de02efb41bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:40.086823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1558bb8-e71c-4f26-a3c7-de02efb41bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:40.087492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1558bb8-e71c-4f26-a3c7-de02efb41bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:40.092989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1558bb8-e71c-4f26-a3c7-de02efb41bfb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:40.115903 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81f611-8143-44c6-b8a4-3e2cacd604bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:40.117826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81f611-8143-44c6-b8a4-3e2cacd604bb service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:40.127888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 946/1922] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:40.129736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1558bb8-e71c-4f26-a3c7-de02efb41bfb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:40.146107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 977/1923] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:40.177944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1558bb8-e71c-4f26-a3c7-de02efb41bfb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:40.192795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da97eab-1f9c-47b9-8cc0-8e6969b32d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:40.193458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97eab-1f9c-47b9-8cc0-8e6969b32d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:40.193741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97eab-1f9c-47b9-8cc0-8e6969b32d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:40.194009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97eab-1f9c-47b9-8cc0-8e6969b32d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:40.194658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da97eab-1f9c-47b9-8cc0-8e6969b32d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:40.199502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da97eab-1f9c-47b9-8cc0-8e6969b32d0b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:40.228406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 947/1924] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:40.233154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81f611-8143-44c6-b8a4-3e2cacd604bb service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:40.253873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81f611-8143-44c6-b8a4-3e2cacd604bb service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:40.263981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2704b8a-c9e6-47e1-bd80-bc44e9541e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:40.264772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2704b8a-c9e6-47e1-bd80-bc44e9541e99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:40.265047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2704b8a-c9e6-47e1-bd80-bc44e9541e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:40.265317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2704b8a-c9e6-47e1-bd80-bc44e9541e99 admin admin] PATH_INFO: `/v3/projects/23ded5c1ba52418ba08ef1f40ae53f59/users/0422338ce99148efa645561fd8e89dbd/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:40.301666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2704b8a-c9e6-47e1-bd80-bc44e9541e99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ded5c1ba52418ba08ef1f40ae53f59, user_id=0422338ce99148efa645561fd8e89dbd, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:40.311357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2704b8a-c9e6-47e1-bd80-bc44e9541e99 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:40.342626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 978/1925] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:40 2022] PUT /identity/v3/projects/23ded5c1ba52418ba08ef1f40ae53f59/users/0422338ce99148efa645561fd8e89dbd/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:40.353109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdf102-9344-42d7-9836-d392a57c9e5a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:40.353392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdf102-9344-42d7-9836-d392a57c9e5a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:40.353661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdf102-9344-42d7-9836-d392a57c9e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:40.412876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-44fdf102-9344-42d7-9836-d392a57c9e5a None None] MFA Rules not processed for user `0422338ce99148efa645561fd8e89dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:40.414123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-44fdf102-9344-42d7-9836-d392a57c9e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:40.477911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:40.512520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 948/1926] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:40.515084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2704b8a-c9e6-47e1-bd80-bc44e9541e99 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:40.646332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2704b8a-c9e6-47e1-bd80-bc44e9541e99 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:40.654430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554a90a-4f93-44da-99af-53a9b5e2feec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:40.655428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554a90a-4f93-44da-99af-53a9b5e2feec admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:40.655996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554a90a-4f93-44da-99af-53a9b5e2feec admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:40.656592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554a90a-4f93-44da-99af-53a9b5e2feec admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:40.657460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554a90a-4f93-44da-99af-53a9b5e2feec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:40.660434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554a90a-4f93-44da-99af-53a9b5e2feec admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:40.703691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 979/1927] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:40 2022] POST /identity/v3/projects => generated 369 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:40.706599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdf102-9344-42d7-9836-d392a57c9e5a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:40.720308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdf102-9344-42d7-9836-d392a57c9e5a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:40.889259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4646f-772e-42ee-a2f9-a6c0473b12eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:40.890918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4646f-772e-42ee-a2f9-a6c0473b12eb admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:40.890918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4646f-772e-42ee-a2f9-a6c0473b12eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:40.890918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4646f-772e-42ee-a2f9-a6c0473b12eb admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:40.891186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4646f-772e-42ee-a2f9-a6c0473b12eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:40.893494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4646f-772e-42ee-a2f9-a6c0473b12eb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:40.942903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 949/1928] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:40 2022] POST /identity/v3/users => generated 291 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:40.946704 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554a90a-4f93-44da-99af-53a9b5e2feec admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:40.969977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554a90a-4f93-44da-99af-53a9b5e2feec admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:40.982507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4646f-772e-42ee-a2f9-a6c0473b12eb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:40.984792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4ff4646f-772e-42ee-a2f9-a6c0473b12eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:40.992083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab2133-ef26-4b2f-9013-aff9f4b57669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:40.992775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2133-ef26-4b2f-9013-aff9f4b57669 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:40.993058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2133-ef26-4b2f-9013-aff9f4b57669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:40.999681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2133-ef26-4b2f-9013-aff9f4b57669 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:41.000100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab2133-ef26-4b2f-9013-aff9f4b57669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:41.004174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab2133-ef26-4b2f-9013-aff9f4b57669 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:41.036867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4646f-772e-42ee-a2f9-a6c0473b12eb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:41.047540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6410d2-03e4-4b94-9566-e1c5fa0b7327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:41.047698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 980/1929] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:41.048525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6410d2-03e4-4b94-9566-e1c5fa0b7327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:41.050122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6410d2-03e4-4b94-9566-e1c5fa0b7327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:41.050545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6410d2-03e4-4b94-9566-e1c5fa0b7327 admin admin] PATH_INFO: `/v3/users/56d90efe060a42678a0a4794013a50ab` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:41.051476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab2133-ef26-4b2f-9013-aff9f4b57669 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:41.052303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6410d2-03e4-4b94-9566-e1c5fa0b7327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d90efe060a42678a0a4794013a50ab)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:41.054470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6410d2-03e4-4b94-9566-e1c5fa0b7327 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:41.067682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab2133-ef26-4b2f-9013-aff9f4b57669 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:41.106065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0c796-7b7a-49f6-ab67-a02914414268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:41.106758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0c796-7b7a-49f6-ab67-a02914414268 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:41.107037 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0c796-7b7a-49f6-ab67-a02914414268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:41.107305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0c796-7b7a-49f6-ab67-a02914414268 admin admin] PATH_INFO: `/v3/projects/e8a41dd736a84d0c8552dff656d0173a/users/b677d29664cf4c1fa020862a93fe8b29/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:41.128097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5e6410d2-03e4-4b94-9566-e1c5fa0b7327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d90efe060a42678a0a4794013a50ab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:41.138245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5e6410d2-03e4-4b94-9566-e1c5fa0b7327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d90efe060a42678a0a4794013a50ab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:41.139108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5e6410d2-03e4-4b94-9566-e1c5fa0b7327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d90efe060a42678a0a4794013a50ab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:41.148896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0c796-7b7a-49f6-ab67-a02914414268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a41dd736a84d0c8552dff656d0173a, user_id=b677d29664cf4c1fa020862a93fe8b29, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:41.156071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5e6410d2-03e4-4b94-9566-e1c5fa0b7327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d90efe060a42678a0a4794013a50ab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:41.164153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0c796-7b7a-49f6-ab67-a02914414268 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:41.186473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5e6410d2-03e4-4b94-9566-e1c5fa0b7327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d90efe060a42678a0a4794013a50ab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:41.199541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 950/1930] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:40 2022] DELETE /identity/v3/users/56d90efe060a42678a0a4794013a50ab => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:41.206102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fd9a0-ff9c-4307-827f-643db4693d31 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:41.207209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fd9a0-ff9c-4307-827f-643db4693d31 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:41.208482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fd9a0-ff9c-4307-827f-643db4693d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:41.210820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 981/1931] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:41 2022] PUT /identity/v3/projects/e8a41dd736a84d0c8552dff656d0173a/users/b677d29664cf4c1fa020862a93fe8b29/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:41.211905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0c796-7b7a-49f6-ab67-a02914414268 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:41.218486 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-31b0c796-7b7a-49f6-ab67-a02914414268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:41.289599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f90fd9a0-ff9c-4307-827f-643db4693d31 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:41.292470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f90fd9a0-ff9c-4307-827f-643db4693d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:41.331293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:41.336432 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0c796-7b7a-49f6-ab67-a02914414268 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:41.488551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:41.532567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2f068-a6df-40ef-9ee7-ec854d2b5e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:41.533198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2f068-a6df-40ef-9ee7-ec854d2b5e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:41.534582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2f068-a6df-40ef-9ee7-ec854d2b5e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:41.534865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2f068-a6df-40ef-9ee7-ec854d2b5e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:41.535632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2f068-a6df-40ef-9ee7-ec854d2b5e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:41.537317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2f068-a6df-40ef-9ee7-ec854d2b5e48 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:41.544396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 951/1932] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:41.573489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 982/1933] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:41.574394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fd9a0-ff9c-4307-827f-643db4693d31 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:41.589246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fd9a0-ff9c-4307-827f-643db4693d31 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:41.626199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44ba5e-c2bf-409f-8cc7-0ef87ffcbe98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:41.627202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44ba5e-c2bf-409f-8cc7-0ef87ffcbe98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:41.627491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44ba5e-c2bf-409f-8cc7-0ef87ffcbe98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:41.627758 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44ba5e-c2bf-409f-8cc7-0ef87ffcbe98 admin admin] PATH_INFO: `/v3/projects/e8a41dd736a84d0c8552dff656d0173a/users/b677d29664cf4c1fa020862a93fe8b29/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:41.645815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44ba5e-c2bf-409f-8cc7-0ef87ffcbe98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a41dd736a84d0c8552dff656d0173a, user_id=b677d29664cf4c1fa020862a93fe8b29, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:41.651260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44ba5e-c2bf-409f-8cc7-0ef87ffcbe98 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:41.662890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2f068-a6df-40ef-9ee7-ec854d2b5e48 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:41.664908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c7b2f068-a6df-40ef-9ee7-ec854d2b5e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:41.673193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 952/1934] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:41 2022] PUT /identity/v3/projects/e8a41dd736a84d0c8552dff656d0173a/users/b677d29664cf4c1fa020862a93fe8b29/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:41.687690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02555b-f73c-4d6f-aefb-569dc37812bf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:41.688158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02555b-f73c-4d6f-aefb-569dc37812bf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:41.689362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02555b-f73c-4d6f-aefb-569dc37812bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:41.746898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2f068-a6df-40ef-9ee7-ec854d2b5e48 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:41.781662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7d02555b-f73c-4d6f-aefb-569dc37812bf None None] MFA Rules not processed for user `b677d29664cf4c1fa020862a93fe8b29`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:41.784275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7d02555b-f73c-4d6f-aefb-569dc37812bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:41.928240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a754114-83aa-492b-beb7-f1e2dbc3960d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:41.933242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a754114-83aa-492b-beb7-f1e2dbc3960d service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:41.933557 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a754114-83aa-492b-beb7-f1e2dbc3960d service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:41.934591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a754114-83aa-492b-beb7-f1e2dbc3960d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:42.036252 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 953/1935] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:42.045247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a754114-83aa-492b-beb7-f1e2dbc3960d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:42.047681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a754114-83aa-492b-beb7-f1e2dbc3960d service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:42.063038 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 983/1936] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:42.074010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02555b-f73c-4d6f-aefb-569dc37812bf None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:42.076314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7d02555b-f73c-4d6f-aefb-569dc37812bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:42.188653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02555b-f73c-4d6f-aefb-569dc37812bf None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:42.194633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e54d92-6efc-4ab1-bf08-18198e66f92b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:42.195391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e54d92-6efc-4ab1-bf08-18198e66f92b service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:42.195747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e54d92-6efc-4ab1-bf08-18198e66f92b service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:42.196071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e54d92-6efc-4ab1-bf08-18198e66f92b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:42.212432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e54d92-6efc-4ab1-bf08-18198e66f92b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:42.214603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e54d92-6efc-4ab1-bf08-18198e66f92b service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:42.232103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 954/1937] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:42.368937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a754114-83aa-492b-beb7-f1e2dbc3960d service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:42.370797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3a754114-83aa-492b-beb7-f1e2dbc3960d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:42.403330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a754114-83aa-492b-beb7-f1e2dbc3960d service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:42.426357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bd427-0791-4cae-9df0-d849026c4e7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:42.429238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd427-0791-4cae-9df0-d849026c4e7c service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:42.430404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd427-0791-4cae-9df0-d849026c4e7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:42.430986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd427-0791-4cae-9df0-d849026c4e7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:42.440789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ad9bd427-0791-4cae-9df0-d849026c4e7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:42.478207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:42.793409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e54d92-6efc-4ab1-bf08-18198e66f92b service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:42.795176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-81e54d92-6efc-4ab1-bf08-18198e66f92b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:42.797053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bd427-0791-4cae-9df0-d849026c4e7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:42.798519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bd427-0791-4cae-9df0-d849026c4e7c service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:42.820958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e54d92-6efc-4ab1-bf08-18198e66f92b service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:42.826529 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 984/1938] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:42.844109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:42.844858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:42.845214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:42.845548 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] PATH_INFO: `/v3/projects/1423d6f63a894abf97d6e8eeb97b259f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:42.847295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1423d6f63a894abf97d6e8eeb97b259f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:42.849395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:42.934125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1423d6f63a894abf97d6e8eeb97b259f', 'request_id': 'req-2b14be3e-b59e-4964-b829-5731f9d7259a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:42.943054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1423d6f63a894abf97d6e8eeb97b259f', 'request_id': 'req-2b14be3e-b59e-4964-b829-5731f9d7259a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:42.950893 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1423d6f63a894abf97d6e8eeb97b259f', 'request_id': 'req-2b14be3e-b59e-4964-b829-5731f9d7259a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:42.952652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:42.988066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] The token cache is being invalidate because project 1423d6f63a894abf97d6e8eeb97b259f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:42.988599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:42.990648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b14be3e-b59e-4964-b829-5731f9d7259a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:42.993904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 955/1939] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:42 2022] DELETE /identity/v3/projects/1423d6f63a894abf97d6e8eeb97b259f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:42.998856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bd427-0791-4cae-9df0-d849026c4e7c service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:43.022587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330de21-42fb-476e-9b2c-398823b9e8d0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:43.022872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330de21-42fb-476e-9b2c-398823b9e8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:43.023149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330de21-42fb-476e-9b2c-398823b9e8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:43.035391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bd427-0791-4cae-9df0-d849026c4e7c service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:43.089333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8330de21-42fb-476e-9b2c-398823b9e8d0 None None] MFA Rules not processed for user `0e76ffd048d240ab94ee7e30b261d3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:43.091273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8330de21-42fb-476e-9b2c-398823b9e8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:43.121686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c51dca-3b5c-4da8-9a10-3e9a3ae220f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:43.122388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c51dca-3b5c-4da8-9a10-3e9a3ae220f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:43.122629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c51dca-3b5c-4da8-9a10-3e9a3ae220f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:43.123150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c51dca-3b5c-4da8-9a10-3e9a3ae220f1 admin admin] PATH_INFO: `/v3/users/c277638d965542eebcde4e45aca265a3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:43.124486 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c51dca-3b5c-4da8-9a10-3e9a3ae220f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c277638d965542eebcde4e45aca265a3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:43.126589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c51dca-3b5c-4da8-9a10-3e9a3ae220f1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:43.201160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 956/1940] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:43.213722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-08c51dca-3b5c-4da8-9a10-3e9a3ae220f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c277638d965542eebcde4e45aca265a3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:43.221780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-08c51dca-3b5c-4da8-9a10-3e9a3ae220f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c277638d965542eebcde4e45aca265a3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:43.222660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-08c51dca-3b5c-4da8-9a10-3e9a3ae220f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c277638d965542eebcde4e45aca265a3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:43.248949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-08c51dca-3b5c-4da8-9a10-3e9a3ae220f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c277638d965542eebcde4e45aca265a3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:43.269731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-08c51dca-3b5c-4da8-9a10-3e9a3ae220f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c277638d965542eebcde4e45aca265a3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:43.279368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 985/1941] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:42 2022] DELETE /identity/v3/users/c277638d965542eebcde4e45aca265a3 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:43.679001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330de21-42fb-476e-9b2c-398823b9e8d0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:43.680679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8330de21-42fb-476e-9b2c-398823b9e8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:43.698148 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330de21-42fb-476e-9b2c-398823b9e8d0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:43.785997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:43.786866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:43.787213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:43.787485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] PATH_INFO: `/v3/projects/b4b1eb30c96a4d7196993641d8e94074` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:43.788813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4b1eb30c96a4d7196993641d8e94074)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:43.791695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:43.911225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4b1eb30c96a4d7196993641d8e94074', 'request_id': 'req-2573c560-9a34-4a20-bb4a-f2949607c0d0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:43.921074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4b1eb30c96a4d7196993641d8e94074', 'request_id': 'req-2573c560-9a34-4a20-bb4a-f2949607c0d0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:43.928581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4b1eb30c96a4d7196993641d8e94074', 'request_id': 'req-2573c560-9a34-4a20-bb4a-f2949607c0d0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:43.949670 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] The token cache is being invalidate because project b4b1eb30c96a4d7196993641d8e94074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:43.949966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:43.950637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2573c560-9a34-4a20-bb4a-f2949607c0d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:43.952476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 957/1942] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:43 2022] DELETE /identity/v3/projects/b4b1eb30c96a4d7196993641d8e94074 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:43.959204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3ac7-35e6-4877-9705-bb24c98129f6 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:43.959503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3ac7-35e6-4877-9705-bb24c98129f6 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:43.959753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3ac7-35e6-4877-9705-bb24c98129f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:43.984999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-93bb3ac7-35e6-4877-9705-bb24c98129f6 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:43.987156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-93bb3ac7-35e6-4877-9705-bb24c98129f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:44.009455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:44.065926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 986/1943] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:44.075759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec239-d3e6-406e-9adc-3e62d2b3fdf7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:44.075759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec239-d3e6-406e-9adc-3e62d2b3fdf7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:44.075759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec239-d3e6-406e-9adc-3e62d2b3fdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:44.102680 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-aedec239-d3e6-406e-9adc-3e62d2b3fdf7 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:44.105231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-aedec239-d3e6-406e-9adc-3e62d2b3fdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:44.116189 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 958/1944] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:30:44.119972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb3ac7-35e6-4877-9705-bb24c98129f6 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:44.132513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb3ac7-35e6-4877-9705-bb24c98129f6 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:44.139734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825304e2-a513-40cd-9d7d-cfe86db7048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:44.140565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825304e2-a513-40cd-9d7d-cfe86db7048d admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:44.140849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825304e2-a513-40cd-9d7d-cfe86db7048d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:44.141209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825304e2-a513-40cd-9d7d-cfe86db7048d admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:44.142192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825304e2-a513-40cd-9d7d-cfe86db7048d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:30:44.142796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825304e2-a513-40cd-9d7d-cfe86db7048d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:44.145117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825304e2-a513-40cd-9d7d-cfe86db7048d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:44.160977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 987/1945] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:30:44.165243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedec239-d3e6-406e-9adc-3e62d2b3fdf7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:44.190982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedec239-d3e6-406e-9adc-3e62d2b3fdf7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:44.200088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83574ed-801e-4440-8bf3-cefebea125cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:44.200749 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83574ed-801e-4440-8bf3-cefebea125cf admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:44.201032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83574ed-801e-4440-8bf3-cefebea125cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:44.201301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83574ed-801e-4440-8bf3-cefebea125cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:44.202043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83574ed-801e-4440-8bf3-cefebea125cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:44.204430 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83574ed-801e-4440-8bf3-cefebea125cf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:44.234735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:44.255127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 959/1946] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:44 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:44.256402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825304e2-a513-40cd-9d7d-cfe86db7048d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:44.269829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825304e2-a513-40cd-9d7d-cfe86db7048d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:44.343414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f913d26-9f93-47ab-9d65-a819148e5a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:44.344129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f913d26-9f93-47ab-9d65-a819148e5a76 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:44.344508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f913d26-9f93-47ab-9d65-a819148e5a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:44.344849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f913d26-9f93-47ab-9d65-a819148e5a76 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:44.345550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f913d26-9f93-47ab-9d65-a819148e5a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:44.347790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f913d26-9f93-47ab-9d65-a819148e5a76 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:44.375070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 988/1947] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:44 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:44.379356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83574ed-801e-4440-8bf3-cefebea125cf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:44.395331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83574ed-801e-4440-8bf3-cefebea125cf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:44.410732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e6c85-fac2-4703-b08c-e146eda3be18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:44.411422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e6c85-fac2-4703-b08c-e146eda3be18 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:44.420115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e6c85-fac2-4703-b08c-e146eda3be18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:44.420688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e6c85-fac2-4703-b08c-e146eda3be18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:44.422473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e6c85-fac2-4703-b08c-e146eda3be18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:44.425013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e6c85-fac2-4703-b08c-e146eda3be18 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:44.454637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 960/1948] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:44.460777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f913d26-9f93-47ab-9d65-a819148e5a76 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:44.488693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f913d26-9f93-47ab-9d65-a819148e5a76 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:44.509238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed3661-498a-412f-be59-7bd59a391fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:44.511020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed3661-498a-412f-be59-7bd59a391fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:44.511281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed3661-498a-412f-be59-7bd59a391fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:44.512841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed3661-498a-412f-be59-7bd59a391fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:44.513614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed3661-498a-412f-be59-7bd59a391fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:44.515256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed3661-498a-412f-be59-7bd59a391fa5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:44.537012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 989/1949] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:44.542569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e6c85-fac2-4703-b08c-e146eda3be18 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:44.554766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e6c85-fac2-4703-b08c-e146eda3be18 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:44.563501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87431f22-6741-49ef-aae8-a746d781e021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:44.564264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87431f22-6741-49ef-aae8-a746d781e021 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:44.565907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87431f22-6741-49ef-aae8-a746d781e021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:44.566202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87431f22-6741-49ef-aae8-a746d781e021 admin admin] PATH_INFO: `/v3/projects/9f40169a694949c381b25adbdfeb06b8/users/6b38a48c07b6454690fb4906d4e0146c/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:44.596208 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87431f22-6741-49ef-aae8-a746d781e021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f40169a694949c381b25adbdfeb06b8, user_id=6b38a48c07b6454690fb4906d4e0146c, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:44.599129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87431f22-6741-49ef-aae8-a746d781e021 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:44.630356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 961/1950] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:44 2022] PUT /identity/v3/projects/9f40169a694949c381b25adbdfeb06b8/users/6b38a48c07b6454690fb4906d4e0146c/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:44.642567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb1e62-7595-4859-8758-114b628251aa None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:44.642845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb1e62-7595-4859-8758-114b628251aa None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:44.643090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb1e62-7595-4859-8758-114b628251aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:44.687326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7ceb1e62-7595-4859-8758-114b628251aa None None] MFA Rules not processed for user `6b38a48c07b6454690fb4906d4e0146c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:44.688611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7ceb1e62-7595-4859-8758-114b628251aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:44.767647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 990/1951] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:44.779838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87431f22-6741-49ef-aae8-a746d781e021 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:44.786103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-87431f22-6741-49ef-aae8-a746d781e021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:44.869714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87431f22-6741-49ef-aae8-a746d781e021 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:44.878089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff927c-370e-479e-9e81-6436552a16e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:44.878711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff927c-370e-479e-9e81-6436552a16e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:44.878987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff927c-370e-479e-9e81-6436552a16e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:44.879321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff927c-370e-479e-9e81-6436552a16e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:44.897343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff927c-370e-479e-9e81-6436552a16e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:44.898878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff927c-370e-479e-9e81-6436552a16e6 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:44.910413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 962/1952] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:30:44.957894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb1e62-7595-4859-8758-114b628251aa None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:45.009568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:45.111623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb1e62-7595-4859-8758-114b628251aa None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:45.115532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:45.116178 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:45.116483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:45.116832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] PATH_INFO: `/v3/users/6b38a48c07b6454690fb4906d4e0146c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:45.118262 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b38a48c07b6454690fb4906d4e0146c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:45.120196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:45.192827 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b38a48c07b6454690fb4906d4e0146c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:45.216452 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b38a48c07b6454690fb4906d4e0146c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:45.216452 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b38a48c07b6454690fb4906d4e0146c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:45.230092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b38a48c07b6454690fb4906d4e0146c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:45.242428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b38a48c07b6454690fb4906d4e0146c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:45.248563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 991/1953] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:44 2022] DELETE /identity/v3/users/6b38a48c07b6454690fb4906d4e0146c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:45.253151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45969d71-9890-4ddf-a1d9-8b8e49870ed8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:45.253654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45969d71-9890-4ddf-a1d9-8b8e49870ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:45.253784 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45969d71-9890-4ddf-a1d9-8b8e49870ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:45.277664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-45969d71-9890-4ddf-a1d9-8b8e49870ed8 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:45.277664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-45969d71-9890-4ddf-a1d9-8b8e49870ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:45.361052 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 963/1954] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:45.371667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:45.373291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:45.460091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce3148-b754-46ca-b656-08d734dbe528 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:45.465927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342fe43-2945-48c3-8223-c08118d12d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:45.466565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342fe43-2945-48c3-8223-c08118d12d3b service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:45.466845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342fe43-2945-48c3-8223-c08118d12d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:45.467119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342fe43-2945-48c3-8223-c08118d12d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:45.478164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342fe43-2945-48c3-8223-c08118d12d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:45.479838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342fe43-2945-48c3-8223-c08118d12d3b service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:45.494332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 992/1955] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:46.171202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45969d71-9890-4ddf-a1d9-8b8e49870ed8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:46.177298 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-45969d71-9890-4ddf-a1d9-8b8e49870ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:46.193345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45969d71-9890-4ddf-a1d9-8b8e49870ed8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:46.222366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:46.223021 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:46.223786 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:46.223786 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] PATH_INFO: `/v3/projects/9f40169a694949c381b25adbdfeb06b8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:46.232842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f40169a694949c381b25adbdfeb06b8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:46.235057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:46.340168 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f40169a694949c381b25adbdfeb06b8', 'request_id': 'req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:46.355331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f40169a694949c381b25adbdfeb06b8', 'request_id': 'req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:46.361304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:46.372252 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f40169a694949c381b25adbdfeb06b8', 'request_id': 'req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:46.397393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] The token cache is being invalidate because project 9f40169a694949c381b25adbdfeb06b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:46.397693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:46.398585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef9d8ccc-0eca-4d63-8af1-70ba18d1afa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:46.402812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 964/1956] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:46 2022] DELETE /identity/v3/projects/9f40169a694949c381b25adbdfeb06b8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:46.407666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886b47-79b0-4a8e-bbd3-36b76861afc9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:46.408157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886b47-79b0-4a8e-bbd3-36b76861afc9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:46.408346 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46886b47-79b0-4a8e-bbd3-36b76861afc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:46.448453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-46886b47-79b0-4a8e-bbd3-36b76861afc9 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:46.453045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-46886b47-79b0-4a8e-bbd3-36b76861afc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:46.529034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:46.562572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 993/1957] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:30:46.570527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6c7cc-d97a-42d3-b9d2-248179beb1b5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:46.571151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6c7cc-d97a-42d3-b9d2-248179beb1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:46.571575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6c7cc-d97a-42d3-b9d2-248179beb1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:46.611272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bdb6c7cc-d97a-42d3-b9d2-248179beb1b5 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:46.625392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bdb6c7cc-d97a-42d3-b9d2-248179beb1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:46.664830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 965/1958] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:46.667557 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46886b47-79b0-4a8e-bbd3-36b76861afc9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:46.686251 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46886b47-79b0-4a8e-bbd3-36b76861afc9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:46.702866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b421a-f33b-4c9f-a28f-37d193ef8bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:46.703505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b421a-f33b-4c9f-a28f-37d193ef8bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:46.703780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b421a-f33b-4c9f-a28f-37d193ef8bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:46.704049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b421a-f33b-4c9f-a28f-37d193ef8bd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:46.704744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b421a-f33b-4c9f-a28f-37d193ef8bd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:30:46.707072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b421a-f33b-4c9f-a28f-37d193ef8bd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:46.711347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b421a-f33b-4c9f-a28f-37d193ef8bd4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:46.769261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 994/1959] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:30:46.772742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6c7cc-d97a-42d3-b9d2-248179beb1b5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:46.788256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6c7cc-d97a-42d3-b9d2-248179beb1b5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:46.804213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b669e-b497-48b6-9178-fe8ce9e013b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:46.805630 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b669e-b497-48b6-9178-fe8ce9e013b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:46.806124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b669e-b497-48b6-9178-fe8ce9e013b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:46.806398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b669e-b497-48b6-9178-fe8ce9e013b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:46.808961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b669e-b497-48b6-9178-fe8ce9e013b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:46.813936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b669e-b497-48b6-9178-fe8ce9e013b0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:46.874061 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 966/1960] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:46 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:46.879352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b421a-f33b-4c9f-a28f-37d193ef8bd4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:46.894655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b421a-f33b-4c9f-a28f-37d193ef8bd4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:46.958030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b669e-b497-48b6-9178-fe8ce9e013b0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:46.959883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cf4b669e-b497-48b6-9178-fe8ce9e013b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:46.994695 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fdf19-0b10-448a-9f1b-ce2eab77c039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:46.995255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fdf19-0b10-448a-9f1b-ce2eab77c039 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:46.995546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fdf19-0b10-448a-9f1b-ce2eab77c039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:46.996122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fdf19-0b10-448a-9f1b-ce2eab77c039 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:46.996580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fdf19-0b10-448a-9f1b-ce2eab77c039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:46.998801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fdf19-0b10-448a-9f1b-ce2eab77c039 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:47.031731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 995/1961] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:46 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:30:47.032773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fdf19-0b10-448a-9f1b-ce2eab77c039 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:47.042558 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b669e-b497-48b6-9178-fe8ce9e013b0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:47.044018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fdf19-0b10-448a-9f1b-ce2eab77c039 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:47.053044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487163d-17b6-45c3-863e-7b0eed9840e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:47.053666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2405-3b44-4985-93bd-4d79519e1b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:47.054323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487163d-17b6-45c3-863e-7b0eed9840e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.054613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487163d-17b6-45c3-863e-7b0eed9840e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.055797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487163d-17b6-45c3-863e-7b0eed9840e0 admin admin] PATH_INFO: `/v3/users/0422338ce99148efa645561fd8e89dbd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.056591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2405-3b44-4985-93bd-4d79519e1b90 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.057032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2405-3b44-4985-93bd-4d79519e1b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.058437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a2405-3b44-4985-93bd-4d79519e1b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.059199 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a2405-3b44-4985-93bd-4d79519e1b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:47.060444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487163d-17b6-45c3-863e-7b0eed9840e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0422338ce99148efa645561fd8e89dbd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:47.062562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487163d-17b6-45c3-863e-7b0eed9840e0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:47.063444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a2405-3b44-4985-93bd-4d79519e1b90 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:47.075638 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 996/1962] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:47.080360 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2405-3b44-4985-93bd-4d79519e1b90 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:47.093042 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a2405-3b44-4985-93bd-4d79519e1b90 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:47.117261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b65a8-6550-443f-9cd2-1c16b5a0b3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:47.117891 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b65a8-6550-443f-9cd2-1c16b5a0b3ff admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.118171 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b65a8-6550-443f-9cd2-1c16b5a0b3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.118443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b65a8-6550-443f-9cd2-1c16b5a0b3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.119104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b65a8-6550-443f-9cd2-1c16b5a0b3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:47.124984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b65a8-6550-443f-9cd2-1c16b5a0b3ff admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:47.158861 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 997/1963] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:47.160070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b65a8-6550-443f-9cd2-1c16b5a0b3ff admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:47.162092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7487163d-17b6-45c3-863e-7b0eed9840e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0422338ce99148efa645561fd8e89dbd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:47.170159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7487163d-17b6-45c3-863e-7b0eed9840e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0422338ce99148efa645561fd8e89dbd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:47.170876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7487163d-17b6-45c3-863e-7b0eed9840e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0422338ce99148efa645561fd8e89dbd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:47.181433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b65a8-6550-443f-9cd2-1c16b5a0b3ff admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:47.188853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7487163d-17b6-45c3-863e-7b0eed9840e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0422338ce99148efa645561fd8e89dbd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:47.199904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d8b03-24d1-4b2b-8a4c-7b9621669a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:47.200959 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d8b03-24d1-4b2b-8a4c-7b9621669a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.201450 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d8b03-24d1-4b2b-8a4c-7b9621669a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.201929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d8b03-24d1-4b2b-8a4c-7b9621669a06 admin admin] PATH_INFO: `/v3/projects/d6663e12901342e79563ae5fb434a8fd/users/26ad8826b4624ef88de4cf8eaf08f236/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.226624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7487163d-17b6-45c3-863e-7b0eed9840e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0422338ce99148efa645561fd8e89dbd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:47.231796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d8b03-24d1-4b2b-8a4c-7b9621669a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6663e12901342e79563ae5fb434a8fd, user_id=26ad8826b4624ef88de4cf8eaf08f236, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:47.235857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 967/1964] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:46 2022] DELETE /identity/v3/users/0422338ce99148efa645561fd8e89dbd => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:47.236106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d8b03-24d1-4b2b-8a4c-7b9621669a06 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:47.253603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680596fd-cb14-4190-af67-7539b1823a4e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.253893 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680596fd-cb14-4190-af67-7539b1823a4e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.254171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680596fd-cb14-4190-af67-7539b1823a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.255060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 998/1965] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:47 2022] PUT /identity/v3/projects/d6663e12901342e79563ae5fb434a8fd/users/26ad8826b4624ef88de4cf8eaf08f236/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:47.266334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96e4a3-e596-4a0a-953d-40846171339b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.266724 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96e4a3-e596-4a0a-953d-40846171339b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.267064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96e4a3-e596-4a0a-953d-40846171339b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.281234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-680596fd-cb14-4190-af67-7539b1823a4e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:47.283014 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-680596fd-cb14-4190-af67-7539b1823a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:47.318889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-cb96e4a3-e596-4a0a-953d-40846171339b None None] MFA Rules not processed for user `26ad8826b4624ef88de4cf8eaf08f236`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:47.334354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cb96e4a3-e596-4a0a-953d-40846171339b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:47.377795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:47.442767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 968/1966] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:47.453454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680596fd-cb14-4190-af67-7539b1823a4e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:47.458969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 999/1967] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:30:47.459473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-680596fd-cb14-4190-af67-7539b1823a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:47.460342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96e4a3-e596-4a0a-953d-40846171339b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:47.462361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cb96e4a3-e596-4a0a-953d-40846171339b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:47.479463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:47.488918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96e4a3-e596-4a0a-953d-40846171339b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:47.499516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aafe32-7141-437b-a7e4-dc68e54f5578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:47.500606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aafe32-7141-437b-a7e4-dc68e54f5578 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.501069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aafe32-7141-437b-a7e4-dc68e54f5578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.501779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aafe32-7141-437b-a7e4-dc68e54f5578 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.502932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aafe32-7141-437b-a7e4-dc68e54f5578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:47.506299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aafe32-7141-437b-a7e4-dc68e54f5578 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:47.546736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680596fd-cb14-4190-af67-7539b1823a4e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:47.551088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e52fa-fff1-4fe7-8a82-229ba99e1037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:47.552155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e52fa-fff1-4fe7-8a82-229ba99e1037 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.552793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e52fa-fff1-4fe7-8a82-229ba99e1037 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.553159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e52fa-fff1-4fe7-8a82-229ba99e1037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.557122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1000/1968] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:47 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:47.558286 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aafe32-7141-437b-a7e4-dc68e54f5578 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:47.564776 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e52fa-fff1-4fe7-8a82-229ba99e1037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:47.566687 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e52fa-fff1-4fe7-8a82-229ba99e1037 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:47.579873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 969/1969] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:30:47.638547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aafe32-7141-437b-a7e4-dc68e54f5578 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:47.644552 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34769060-9e02-4300-8e57-88692f8c6384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:47.645260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34769060-9e02-4300-8e57-88692f8c6384 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.645539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34769060-9e02-4300-8e57-88692f8c6384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.645818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34769060-9e02-4300-8e57-88692f8c6384 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.646462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34769060-9e02-4300-8e57-88692f8c6384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:47.648499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34769060-9e02-4300-8e57-88692f8c6384 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:47.688172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1001/1970] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:47 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:47.692446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e52fa-fff1-4fe7-8a82-229ba99e1037 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:47.708191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e52fa-fff1-4fe7-8a82-229ba99e1037 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:47.715394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad0be-4c84-469d-b986-7a075ad6d361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:47.716134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad0be-4c84-469d-b986-7a075ad6d361 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.716550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad0be-4c84-469d-b986-7a075ad6d361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.716921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad0be-4c84-469d-b986-7a075ad6d361 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.718611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bad0be-4c84-469d-b986-7a075ad6d361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:47.722382 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bad0be-4c84-469d-b986-7a075ad6d361 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:47.750469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 970/1971] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:47.751697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34769060-9e02-4300-8e57-88692f8c6384 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:47.762292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34769060-9e02-4300-8e57-88692f8c6384 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:47.768504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a71a3-f63b-40c3-8c97-6e7fc6a0dc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:47.769116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a71a3-f63b-40c3-8c97-6e7fc6a0dc6c admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.769395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a71a3-f63b-40c3-8c97-6e7fc6a0dc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.769665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a71a3-f63b-40c3-8c97-6e7fc6a0dc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.770499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a71a3-f63b-40c3-8c97-6e7fc6a0dc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:47.771903 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a71a3-f63b-40c3-8c97-6e7fc6a0dc6c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:47.782740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1002/1972] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:47.787383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad0be-4c84-469d-b986-7a075ad6d361 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:47.800417 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad0be-4c84-469d-b986-7a075ad6d361 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:47.810796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd5743-cdb3-4f97-bb50-9d33589867fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:47.811709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd5743-cdb3-4f97-bb50-9d33589867fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.812808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd5743-cdb3-4f97-bb50-9d33589867fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.813406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd5743-cdb3-4f97-bb50-9d33589867fa admin admin] PATH_INFO: `/v3/projects/6160361f281341edaf52cbcb443b58c1/users/7f0d2ced677a43448e7bd040f6994fdc/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.832262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd5743-cdb3-4f97-bb50-9d33589867fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6160361f281341edaf52cbcb443b58c1, user_id=7f0d2ced677a43448e7bd040f6994fdc, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:47.835620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd5743-cdb3-4f97-bb50-9d33589867fa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:47.865469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 971/1973] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:47 2022] PUT /identity/v3/projects/6160361f281341edaf52cbcb443b58c1/users/7f0d2ced677a43448e7bd040f6994fdc/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:47.869004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e0e81-fb93-4337-8893-6b524195416b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:47.869315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e0e81-fb93-4337-8893-6b524195416b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:47.869606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e0e81-fb93-4337-8893-6b524195416b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:47.901246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-9e4e0e81-fb93-4337-8893-6b524195416b None None] MFA Rules not processed for user `7f0d2ced677a43448e7bd040f6994fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:47.902498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9e4e0e81-fb93-4337-8893-6b524195416b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:47.986340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1003/1974] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:47.991974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd5743-cdb3-4f97-bb50-9d33589867fa admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:48.068467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd5743-cdb3-4f97-bb50-9d33589867fa admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:48.073412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a2d3a-0885-43b5-956b-3f33f3d03d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:48.074098 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2d3a-0885-43b5-956b-3f33f3d03d59 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:48.074449 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2d3a-0885-43b5-956b-3f33f3d03d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:48.074817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2d3a-0885-43b5-956b-3f33f3d03d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:48.075528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a2d3a-0885-43b5-956b-3f33f3d03d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:48.077555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a2d3a-0885-43b5-956b-3f33f3d03d59 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:48.113083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 972/1975] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:47 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:48.114247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e0e81-fb93-4337-8893-6b524195416b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:48.126752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e0e81-fb93-4337-8893-6b524195416b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:48.128223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a2d3a-0885-43b5-956b-3f33f3d03d59 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:48.130996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f15a2d3a-0885-43b5-956b-3f33f3d03d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:48.219199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:48.224716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:48.254600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b68ed-0662-4236-a715-ccb27ad6a967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:48.256712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b68ed-0662-4236-a715-ccb27ad6a967 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:48.256968 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b68ed-0662-4236-a715-ccb27ad6a967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:48.257253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b68ed-0662-4236-a715-ccb27ad6a967 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:48.259129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b68ed-0662-4236-a715-ccb27ad6a967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:48.262759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b68ed-0662-4236-a715-ccb27ad6a967 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:48.296413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1004/1976] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:48 2022] POST /identity/v3/users => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:48.303499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b68ed-0662-4236-a715-ccb27ad6a967 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:48.318298 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b68ed-0662-4236-a715-ccb27ad6a967 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:48.319454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a2d3a-0885-43b5-956b-3f33f3d03d59 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:48.329801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d56b0-bab4-42d3-8832-c5e3c50ff266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:48.330872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:48.331821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:48.332361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:48.332880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] PATH_INFO: `/v3/projects/23ded5c1ba52418ba08ef1f40ae53f59` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:48.333752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d56b0-bab4-42d3-8832-c5e3c50ff266 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:48.334868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d56b0-bab4-42d3-8832-c5e3c50ff266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:48.335344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d56b0-bab4-42d3-8832-c5e3c50ff266 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:48.336172 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ded5c1ba52418ba08ef1f40ae53f59)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:48.336532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d56b0-bab4-42d3-8832-c5e3c50ff266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:48.339750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d56b0-bab4-42d3-8832-c5e3c50ff266 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:48.342058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:48.355908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1005/1977] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:48.361723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d56b0-bab4-42d3-8832-c5e3c50ff266 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:48.379597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d56b0-bab4-42d3-8832-c5e3c50ff266 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:48.398551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010c4b9-991d-4015-94d1-5e58e263ab6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:48.399238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010c4b9-991d-4015-94d1-5e58e263ab6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:48.402496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010c4b9-991d-4015-94d1-5e58e263ab6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:48.408943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010c4b9-991d-4015-94d1-5e58e263ab6a admin admin] PATH_INFO: `/v3/projects/aa419d9d4c604daabaf1e957f1fd5ca4/users/e9a36174c9ac4aa3934fbdf073442287/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:48.439003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010c4b9-991d-4015-94d1-5e58e263ab6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa419d9d4c604daabaf1e957f1fd5ca4, user_id=e9a36174c9ac4aa3934fbdf073442287, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:48.445054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010c4b9-991d-4015-94d1-5e58e263ab6a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:48.457625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ded5c1ba52418ba08ef1f40ae53f59', 'request_id': 'req-cdffec3c-f09e-4909-a5bb-d54afe016cac'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:48.465737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ded5c1ba52418ba08ef1f40ae53f59', 'request_id': 'req-cdffec3c-f09e-4909-a5bb-d54afe016cac'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:48.467663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1006/1978] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:48 2022] PUT /identity/v3/projects/aa419d9d4c604daabaf1e957f1fd5ca4/users/e9a36174c9ac4aa3934fbdf073442287/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:48.472942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010c4b9-991d-4015-94d1-5e58e263ab6a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:48.479321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ded5c1ba52418ba08ef1f40ae53f59', 'request_id': 'req-cdffec3c-f09e-4909-a5bb-d54afe016cac'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:48.505040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] The token cache is being invalidate because project 23ded5c1ba52418ba08ef1f40ae53f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:48.505910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:48.506811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:48.514798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 973/1979] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:48 2022] DELETE /identity/v3/projects/23ded5c1ba52418ba08ef1f40ae53f59 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:48.520032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:48.521627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:48.602333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010c4b9-991d-4015-94d1-5e58e263ab6a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:48.619510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c704d2f-b0e8-4939-90b2-5e243520f424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:48.620150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c704d2f-b0e8-4939-90b2-5e243520f424 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:48.620456 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c704d2f-b0e8-4939-90b2-5e243520f424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:48.620723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c704d2f-b0e8-4939-90b2-5e243520f424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:48.621471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c704d2f-b0e8-4939-90b2-5e243520f424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:48.623318 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c704d2f-b0e8-4939-90b2-5e243520f424 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:48.631347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffec3c-f09e-4909-a5bb-d54afe016cac admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:48.636581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:48.637223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:48.637497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:48.637769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] PATH_INFO: `/v3/users/b677d29664cf4c1fa020862a93fe8b29` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:48.638913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b677d29664cf4c1fa020862a93fe8b29)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:48.639305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1007/1980] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:48.640921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:48.646521 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c704d2f-b0e8-4939-90b2-5e243520f424 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:48.651863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1c704d2f-b0e8-4939-90b2-5e243520f424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:48.678474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c704d2f-b0e8-4939-90b2-5e243520f424 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:48.693465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0c91d-9393-429c-9734-361a660dc7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:48.694172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0c91d-9393-429c-9734-361a660dc7e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:48.695865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0c91d-9393-429c-9734-361a660dc7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:48.696240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0c91d-9393-429c-9734-361a660dc7e7 admin admin] PATH_INFO: `/v3/projects/aa419d9d4c604daabaf1e957f1fd5ca4/users/e9a36174c9ac4aa3934fbdf073442287/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:48.701281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0c91d-9393-429c-9734-361a660dc7e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa419d9d4c604daabaf1e957f1fd5ca4, user_id=e9a36174c9ac4aa3934fbdf073442287, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:48.709000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0c91d-9393-429c-9734-361a660dc7e7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:48.737627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1008/1981] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:48 2022] PUT /identity/v3/projects/aa419d9d4c604daabaf1e957f1fd5ca4/users/e9a36174c9ac4aa3934fbdf073442287/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:48.749206 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b677d29664cf4c1fa020862a93fe8b29'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:48.750729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3efa67-56de-4631-bb91-5fa7d932f0a2 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:48.751064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3efa67-56de-4631-bb91-5fa7d932f0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:48.751351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3efa67-56de-4631-bb91-5fa7d932f0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:48.763470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b677d29664cf4c1fa020862a93fe8b29'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:48.764109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b677d29664cf4c1fa020862a93fe8b29'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:48.776137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b677d29664cf4c1fa020862a93fe8b29'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:48.779636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:48.798865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fa3efa67-56de-4631-bb91-5fa7d932f0a2 None None] MFA Rules not processed for user `e9a36174c9ac4aa3934fbdf073442287`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:48.800996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fa3efa67-56de-4631-bb91-5fa7d932f0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:48.804950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b677d29664cf4c1fa020862a93fe8b29'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:48.816440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 974/1982] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:48 2022] DELETE /identity/v3/users/b677d29664cf4c1fa020862a93fe8b29 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:48.915246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1009/1983] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:30:48.929892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:48.932405 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:49.018450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8c6a6-f914-4fdd-a326-94ab4e9cac2e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:49.022983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c612452-2147-4cc3-a870-43ef218cb311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:49.023605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c612452-2147-4cc3-a870-43ef218cb311 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:49.023885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c612452-2147-4cc3-a870-43ef218cb311 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:49.024155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c612452-2147-4cc3-a870-43ef218cb311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:49.026699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0c612452-2147-4cc3-a870-43ef218cb311 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:49.141399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c612452-2147-4cc3-a870-43ef218cb311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:49.144166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c612452-2147-4cc3-a870-43ef218cb311 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:49.155048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 975/1984] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:49.246845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3efa67-56de-4631-bb91-5fa7d932f0a2 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:49.248842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fa3efa67-56de-4631-bb91-5fa7d932f0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:49.266803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:49.346914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3efa67-56de-4631-bb91-5fa7d932f0a2 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:49.441310 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:49.442156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:49.442438 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:49.442708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] PATH_INFO: `/v3/projects/e8a41dd736a84d0c8552dff656d0173a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:49.443956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a41dd736a84d0c8552dff656d0173a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:49.446137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:49.507344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a41dd736a84d0c8552dff656d0173a', 'request_id': 'req-0130e6d9-38e8-4e25-873f-eafb056d8255'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:49.514960 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a41dd736a84d0c8552dff656d0173a', 'request_id': 'req-0130e6d9-38e8-4e25-873f-eafb056d8255'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:49.524464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a41dd736a84d0c8552dff656d0173a', 'request_id': 'req-0130e6d9-38e8-4e25-873f-eafb056d8255'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:49.543660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] The token cache is being invalidate because project e8a41dd736a84d0c8552dff656d0173a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:49.543954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:49.544624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0130e6d9-38e8-4e25-873f-eafb056d8255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:49.546143 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1010/1985] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:49 2022] DELETE /identity/v3/projects/e8a41dd736a84d0c8552dff656d0173a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:49.551110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00131b5f-28e7-4fdf-ae88-8ca3acd8b6e2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:49.551651 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00131b5f-28e7-4fdf-ae88-8ca3acd8b6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:49.551902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00131b5f-28e7-4fdf-ae88-8ca3acd8b6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:49.575446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-00131b5f-28e7-4fdf-ae88-8ca3acd8b6e2 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:49.576647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-00131b5f-28e7-4fdf-ae88-8ca3acd8b6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:49.641012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 976/1986] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:49.649607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc6e39-7019-4385-be41-ff221be465e5 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:49.649893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc6e39-7019-4385-be41-ff221be465e5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:49.650194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc6e39-7019-4385-be41-ff221be465e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:49.670671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a5cc6e39-7019-4385-be41-ff221be465e5 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:49.671955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a5cc6e39-7019-4385-be41-ff221be465e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:49.680863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1011/1987] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:49.685373 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00131b5f-28e7-4fdf-ae88-8ca3acd8b6e2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:49.695437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00131b5f-28e7-4fdf-ae88-8ca3acd8b6e2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:49.699997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c25124-9882-4196-a247-e2045fea9b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:49.700623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c25124-9882-4196-a247-e2045fea9b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:49.700908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c25124-9882-4196-a247-e2045fea9b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:49.701172 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c25124-9882-4196-a247-e2045fea9b88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:49.701812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c25124-9882-4196-a247-e2045fea9b88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:30:49.702111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c25124-9882-4196-a247-e2045fea9b88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:49.703693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c25124-9882-4196-a247-e2045fea9b88 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:49.715717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 977/1988] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:30:49.720005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc6e39-7019-4385-be41-ff221be465e5 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:49.730956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc6e39-7019-4385-be41-ff221be465e5 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:49.735685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aca0f4-7685-4102-ad69-8b6a7d851fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:49.736302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aca0f4-7685-4102-ad69-8b6a7d851fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:49.736609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aca0f4-7685-4102-ad69-8b6a7d851fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:49.736876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aca0f4-7685-4102-ad69-8b6a7d851fa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:49.737498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aca0f4-7685-4102-ad69-8b6a7d851fa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:49.739846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aca0f4-7685-4102-ad69-8b6a7d851fa0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:49.770016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1012/1989] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:49 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:49.771298 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c25124-9882-4196-a247-e2045fea9b88 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:49.781488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c25124-9882-4196-a247-e2045fea9b88 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:49.803273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:49.845114 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33985939-b02c-427d-8a55-e67a12676da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:49.846287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33985939-b02c-427d-8a55-e67a12676da6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:49.846600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33985939-b02c-427d-8a55-e67a12676da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:49.846898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33985939-b02c-427d-8a55-e67a12676da6 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:49.847529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33985939-b02c-427d-8a55-e67a12676da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:49.849532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33985939-b02c-427d-8a55-e67a12676da6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:49.871051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 978/1990] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:49 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:49.875378 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aca0f4-7685-4102-ad69-8b6a7d851fa0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:49.886356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aca0f4-7685-4102-ad69-8b6a7d851fa0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:49.891712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d50ef4-0628-408b-a758-863a0b42246a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:49.891848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d50ef4-0628-408b-a758-863a0b42246a admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:49.892025 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d50ef4-0628-408b-a758-863a0b42246a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:49.892298 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d50ef4-0628-408b-a758-863a0b42246a admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:49.893512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d50ef4-0628-408b-a758-863a0b42246a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:49.895089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d50ef4-0628-408b-a758-863a0b42246a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:49.905765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1013/1991] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:49.910366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33985939-b02c-427d-8a55-e67a12676da6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:49.921102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33985939-b02c-427d-8a55-e67a12676da6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:49.926291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890fd20-70cf-4443-bd53-45c5434156e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:49.926884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890fd20-70cf-4443-bd53-45c5434156e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:49.927166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890fd20-70cf-4443-bd53-45c5434156e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:49.927430 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890fd20-70cf-4443-bd53-45c5434156e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:49.928062 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890fd20-70cf-4443-bd53-45c5434156e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:49.929708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890fd20-70cf-4443-bd53-45c5434156e8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:49.940036 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 979/1992] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:30:49.944086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d50ef4-0628-408b-a758-863a0b42246a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:49.954562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d50ef4-0628-408b-a758-863a0b42246a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:49.959602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67384-69f4-466c-bd49-19403829e11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:49.960242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67384-69f4-466c-bd49-19403829e11d admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:49.960553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67384-69f4-466c-bd49-19403829e11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:49.961153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67384-69f4-466c-bd49-19403829e11d admin admin] PATH_INFO: `/v3/projects/04e31b183c914d77a8bcc34b214ffa8b/users/05dfecc5811845e4922d0e47db644552/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:49.964226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:49.977585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b67384-69f4-466c-bd49-19403829e11d admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e31b183c914d77a8bcc34b214ffa8b, user_id=05dfecc5811845e4922d0e47db644552, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:49.980836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b67384-69f4-466c-bd49-19403829e11d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:49.994259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1014/1993] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:49 2022] PUT /identity/v3/projects/04e31b183c914d77a8bcc34b214ffa8b/users/05dfecc5811845e4922d0e47db644552/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:49.999770 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476272b-1b3f-4886-b3d3-db82866dd884 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:50.000077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476272b-1b3f-4886-b3d3-db82866dd884 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:50.000327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476272b-1b3f-4886-b3d3-db82866dd884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:50.022064 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9476272b-1b3f-4886-b3d3-db82866dd884 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:50.023290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9476272b-1b3f-4886-b3d3-db82866dd884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:50.082236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 980/1994] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:50.091965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67384-69f4-466c-bd49-19403829e11d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:50.093948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-75b67384-69f4-466c-bd49-19403829e11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:50.153050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67384-69f4-466c-bd49-19403829e11d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:50.156675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ddc2f-5aec-470e-a786-df5ff82623a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:50.157280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ddc2f-5aec-470e-a786-df5ff82623a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:50.157555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ddc2f-5aec-470e-a786-df5ff82623a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:50.157823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ddc2f-5aec-470e-a786-df5ff82623a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:50.168765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ddc2f-5aec-470e-a786-df5ff82623a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:50.170228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ddc2f-5aec-470e-a786-df5ff82623a1 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:50.181349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1015/1995] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:50.801162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476272b-1b3f-4886-b3d3-db82866dd884 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:50.802841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9476272b-1b3f-4886-b3d3-db82866dd884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:50.873334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f819342-e542-4d4d-8616-ef6bca8cf944 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:50.874236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f819342-e542-4d4d-8616-ef6bca8cf944 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:50.874660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f819342-e542-4d4d-8616-ef6bca8cf944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:50.887743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476272b-1b3f-4886-b3d3-db82866dd884 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:50.892955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd062aa-a378-46d1-b24e-c5a289b3cbcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:50.893708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd062aa-a378-46d1-b24e-c5a289b3cbcb service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:50.893979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd062aa-a378-46d1-b24e-c5a289b3cbcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:50.894389 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd062aa-a378-46d1-b24e-c5a289b3cbcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:50.896518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2cd062aa-a378-46d1-b24e-c5a289b3cbcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:50.912712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8f819342-e542-4d4d-8616-ef6bca8cf944 None None] MFA Rules not processed for user `05dfecc5811845e4922d0e47db644552`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:50.914323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8f819342-e542-4d4d-8616-ef6bca8cf944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:51.023183 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1016/1996] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:30:51.028077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f819342-e542-4d4d-8616-ef6bca8cf944 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:51.031400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd062aa-a378-46d1-b24e-c5a289b3cbcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:51.032867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd062aa-a378-46d1-b24e-c5a289b3cbcb service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:51.037338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:51.059057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 981/1997] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:51.073355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f819342-e542-4d4d-8616-ef6bca8cf944 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:51.085387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff31ada-6c64-45fd-98bb-4aee5f050ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:51.086030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff31ada-6c64-45fd-98bb-4aee5f050ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:51.086300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff31ada-6c64-45fd-98bb-4aee5f050ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:51.086567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff31ada-6c64-45fd-98bb-4aee5f050ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:51.087202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff31ada-6c64-45fd-98bb-4aee5f050ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:51.089148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff31ada-6c64-45fd-98bb-4aee5f050ab8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:51.160987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1017/1998] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:51 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:51.161602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd062aa-a378-46d1-b24e-c5a289b3cbcb service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:51.178122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd062aa-a378-46d1-b24e-c5a289b3cbcb service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:51.269695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946d40e-7dc0-4ef2-a1dc-55c0f933e425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:51.270313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946d40e-7dc0-4ef2-a1dc-55c0f933e425 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:51.270587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946d40e-7dc0-4ef2-a1dc-55c0f933e425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:51.270856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946d40e-7dc0-4ef2-a1dc-55c0f933e425 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:51.271478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946d40e-7dc0-4ef2-a1dc-55c0f933e425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:51.273515 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946d40e-7dc0-4ef2-a1dc-55c0f933e425 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:51.297130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 982/1999] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:51 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:30:51.302317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff31ada-6c64-45fd-98bb-4aee5f050ab8 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:51.312515 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff31ada-6c64-45fd-98bb-4aee5f050ab8 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:51.318119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3c8a1-5269-4253-aaec-c11f3662ac91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:51.318733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3c8a1-5269-4253-aaec-c11f3662ac91 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:51.319008 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3c8a1-5269-4253-aaec-c11f3662ac91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:51.319280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3c8a1-5269-4253-aaec-c11f3662ac91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:51.319925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3c8a1-5269-4253-aaec-c11f3662ac91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:51.321556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3c8a1-5269-4253-aaec-c11f3662ac91 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:51.336197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1018/2000] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:30:51.338833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946d40e-7dc0-4ef2-a1dc-55c0f933e425 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:51.354446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946d40e-7dc0-4ef2-a1dc-55c0f933e425 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:51.361171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10e4b4-1a92-4dff-b7ff-b4038d55a734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:51.361921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10e4b4-1a92-4dff-b7ff-b4038d55a734 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:51.362801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10e4b4-1a92-4dff-b7ff-b4038d55a734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:51.363076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10e4b4-1a92-4dff-b7ff-b4038d55a734 admin admin] PATH_INFO: `/v3/projects/3f6a23fdbc5c4786be3c1b8db7aee897/users/85b1c840a9ae4641a38c7c29475ca1e8/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:51.382833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10e4b4-1a92-4dff-b7ff-b4038d55a734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6a23fdbc5c4786be3c1b8db7aee897, user_id=85b1c840a9ae4641a38c7c29475ca1e8, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:51.387509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10e4b4-1a92-4dff-b7ff-b4038d55a734 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:51.410256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 983/2001] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:51 2022] PUT /identity/v3/projects/3f6a23fdbc5c4786be3c1b8db7aee897/users/85b1c840a9ae4641a38c7c29475ca1e8/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:51.414398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3c8a1-5269-4253-aaec-c11f3662ac91 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:51.426739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3c8a1-5269-4253-aaec-c11f3662ac91 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:51.476724 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:51.512109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703575a0-d567-461e-9f37-ea2d0bc51f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:51.512830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703575a0-d567-461e-9f37-ea2d0bc51f96 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:51.513348 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703575a0-d567-461e-9f37-ea2d0bc51f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:51.513776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703575a0-d567-461e-9f37-ea2d0bc51f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:51.514429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703575a0-d567-461e-9f37-ea2d0bc51f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:51.516024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703575a0-d567-461e-9f37-ea2d0bc51f96 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:51.536293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1019/2002] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:30:51.536927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10e4b4-1a92-4dff-b7ff-b4038d55a734 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:51.548209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10e4b4-1a92-4dff-b7ff-b4038d55a734 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:51.553062 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7031863-c3d3-4f13-abe4-f9c3c614bc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:51.553635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031863-c3d3-4f13-abe4-f9c3c614bc7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:51.553915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031863-c3d3-4f13-abe4-f9c3c614bc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:51.554337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031863-c3d3-4f13-abe4-f9c3c614bc7a admin admin] PATH_INFO: `/v3/projects/3f6a23fdbc5c4786be3c1b8db7aee897/users/85b1c840a9ae4641a38c7c29475ca1e8/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:51.557605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7031863-c3d3-4f13-abe4-f9c3c614bc7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6a23fdbc5c4786be3c1b8db7aee897, user_id=85b1c840a9ae4641a38c7c29475ca1e8, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:51.560808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7031863-c3d3-4f13-abe4-f9c3c614bc7a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:51.574467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 984/2003] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:51 2022] PUT /identity/v3/projects/3f6a23fdbc5c4786be3c1b8db7aee897/users/85b1c840a9ae4641a38c7c29475ca1e8/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:51.930100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703575a0-d567-461e-9f37-ea2d0bc51f96 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:52.008954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703575a0-d567-461e-9f37-ea2d0bc51f96 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:52.097228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ea2be-f8fd-4168-9598-05a9038c5b8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:52.097814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea2be-f8fd-4168-9598-05a9038c5b8e service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:52.098114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea2be-f8fd-4168-9598-05a9038c5b8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:52.098373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ea2be-f8fd-4168-9598-05a9038c5b8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:52.100655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7f2ea2be-f8fd-4168-9598-05a9038c5b8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:52.202002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ea2be-f8fd-4168-9598-05a9038c5b8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:52.203953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ea2be-f8fd-4168-9598-05a9038c5b8e service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:52.219876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1020/2004] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:52.461143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3a81a-58f5-444b-ad6f-62b2c76296d1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:52.461772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3a81a-58f5-444b-ad6f-62b2c76296d1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:52.461772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3a81a-58f5-444b-ad6f-62b2c76296d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:52.496340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f2d3a81a-58f5-444b-ad6f-62b2c76296d1 None None] MFA Rules not processed for user `85b1c840a9ae4641a38c7c29475ca1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:52.497927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f2d3a81a-58f5-444b-ad6f-62b2c76296d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:52.585587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 985/2005] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:52.597800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ea2be-f8fd-4168-9598-05a9038c5b8e service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:52.599235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7f2ea2be-f8fd-4168-9598-05a9038c5b8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:52.616113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ea2be-f8fd-4168-9598-05a9038c5b8e service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:52.620264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbfa6-2710-493a-9cbe-25cc06addd8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:52.620931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbfa6-2710-493a-9cbe-25cc06addd8e service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:52.621252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbfa6-2710-493a-9cbe-25cc06addd8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:52.621482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbfa6-2710-493a-9cbe-25cc06addd8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:52.678377 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:52.692652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbfa6-2710-493a-9cbe-25cc06addd8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:52.694095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbfa6-2710-493a-9cbe-25cc06addd8e service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:52.704574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1021/2006] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:54.915800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06ec08-6c02-4d17-967d-e47f812c7673 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:54.916306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06ec08-6c02-4d17-967d-e47f812c7673 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:54.916449 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06ec08-6c02-4d17-967d-e47f812c7673 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:54.917779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 986/2007] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:30:54.950003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd10b3a-5b29-406d-ac60-8f1ffc08c845 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:54.950497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd10b3a-5b29-406d-ac60-8f1ffc08c845 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:54.950599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd10b3a-5b29-406d-ac60-8f1ffc08c845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:54.974522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0dd10b3a-5b29-406d-ac60-8f1ffc08c845 None None] MFA Rules not processed for user `0a50fd417ae94859bb713b8b2066a9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:54.975837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0dd10b3a-5b29-406d-ac60-8f1ffc08c845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:54.992059 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06ec08-6c02-4d17-967d-e47f812c7673 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:54.994200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cd06ec08-6c02-4d17-967d-e47f812c7673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:55.060057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:55.077463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1022/2008] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:30:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:55.085263 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab786b-d284-450d-aa9f-194ae41d3664 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.087302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab786b-d284-450d-aa9f-194ae41d3664 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.087302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab786b-d284-450d-aa9f-194ae41d3664 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.106301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1023/2009] 10.222.0.17 () {54 vars in 961 bytes} [Fri Sep 30 19:30:55 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 19:30:55.106301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb610e7-0e4f-4f06-b6c6-f6debc31434e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.106301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb610e7-0e4f-4f06-b6c6-f6debc31434e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.106301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb610e7-0e4f-4f06-b6c6-f6debc31434e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.109952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06ec08-6c02-4d17-967d-e47f812c7673 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:55.116924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:55.117646 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.118309 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.118614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] PATH_INFO: `/v3/users/f1bb718f9153493ca6dda343d461c5ab` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.120118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1bb718f9153493ca6dda343d461c5ab)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:55.123268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:55.131845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8eb610e7-0e4f-4f06-b6c6-f6debc31434e None None] MFA Rules not processed for user `0a50fd417ae94859bb713b8b2066a9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:55.133240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8eb610e7-0e4f-4f06-b6c6-f6debc31434e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:55.145946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1024/2010] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Sep 30 19:30:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:55.156923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb610e7-0e4f-4f06-b6c6-f6debc31434e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:55.170414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb610e7-0e4f-4f06-b6c6-f6debc31434e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:55.176874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858bac4-3090-4b3d-90b8-95e67824036d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:55.177484 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858bac4-3090-4b3d-90b8-95e67824036d service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.178175 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858bac4-3090-4b3d-90b8-95e67824036d service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.178505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858bac4-3090-4b3d-90b8-95e67824036d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.183188 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1bb718f9153493ca6dda343d461c5ab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:55.191338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1bb718f9153493ca6dda343d461c5ab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:55.192154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1bb718f9153493ca6dda343d461c5ab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:55.195576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858bac4-3090-4b3d-90b8-95e67824036d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:55.196768 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858bac4-3090-4b3d-90b8-95e67824036d service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:55.205994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1bb718f9153493ca6dda343d461c5ab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:55.208090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1025/2011] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:30:55.221673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1bb718f9153493ca6dda343d461c5ab'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:55.229900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 987/2012] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:54 2022] DELETE /identity/v3/users/f1bb718f9153493ca6dda343d461c5ab => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:55.278561 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858bac4-3090-4b3d-90b8-95e67824036d service placement] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:55.288009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858bac4-3090-4b3d-90b8-95e67824036d service placement] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:55.347579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e2289-5239-4129-8879-31059ee54511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:55.348182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e2289-5239-4129-8879-31059ee54511 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.348479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e2289-5239-4129-8879-31059ee54511 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.348752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e2289-5239-4129-8879-31059ee54511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.350760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bd3e2289-5239-4129-8879-31059ee54511 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:55.419467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:55.421009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:55.432163 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e2289-5239-4129-8879-31059ee54511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:55.433840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e2289-5239-4129-8879-31059ee54511 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:55.447289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1026/2013] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:55.503939 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b54de2-2405-45c5-9078-59ef32b3a8ef admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:55.510718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:55.511430 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.511782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.512187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] PATH_INFO: `/v3/projects/d9758637e30e4089b394ed6ea5c2cf3f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.513400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9758637e30e4089b394ed6ea5c2cf3f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:55.515775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:55.572914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9758637e30e4089b394ed6ea5c2cf3f', 'request_id': 'req-258e9de0-f775-4b74-be11-18fcdd1554c2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:55.579519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:55.581909 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9758637e30e4089b394ed6ea5c2cf3f', 'request_id': 'req-258e9de0-f775-4b74-be11-18fcdd1554c2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:55.592783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9758637e30e4089b394ed6ea5c2cf3f', 'request_id': 'req-258e9de0-f775-4b74-be11-18fcdd1554c2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:55.607540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] The token cache is being invalidate because project d9758637e30e4089b394ed6ea5c2cf3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:55.608178 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:55.609099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-258e9de0-f775-4b74-be11-18fcdd1554c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:55.610862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 988/2014] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:55 2022] DELETE /identity/v3/projects/d9758637e30e4089b394ed6ea5c2cf3f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:55.616215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172aa8c2-1f43-4758-acd8-fa53d87c7eff None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.616631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172aa8c2-1f43-4758-acd8-fa53d87c7eff None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.616949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172aa8c2-1f43-4758-acd8-fa53d87c7eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.638201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-172aa8c2-1f43-4758-acd8-fa53d87c7eff None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:55.639423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-172aa8c2-1f43-4758-acd8-fa53d87c7eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:55.647651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:55.720359 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1027/2015] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:55.721984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e779d59-317d-4e0f-96ff-e7fc9695c6e9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.722266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e779d59-317d-4e0f-96ff-e7fc9695c6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.722542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e779d59-317d-4e0f-96ff-e7fc9695c6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.742531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4e779d59-317d-4e0f-96ff-e7fc9695c6e9 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:55.743757 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4e779d59-317d-4e0f-96ff-e7fc9695c6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:55.752766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 989/2016] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:55.757158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172aa8c2-1f43-4758-acd8-fa53d87c7eff None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:55.768057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172aa8c2-1f43-4758-acd8-fa53d87c7eff None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:55.772854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c12c90-4edd-402f-9911-b2e37b0efe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:55.773460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c12c90-4edd-402f-9911-b2e37b0efe01 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.773733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c12c90-4edd-402f-9911-b2e37b0efe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.773996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c12c90-4edd-402f-9911-b2e37b0efe01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.774650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c12c90-4edd-402f-9911-b2e37b0efe01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:30:55.774937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c12c90-4edd-402f-9911-b2e37b0efe01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:55.776543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c12c90-4edd-402f-9911-b2e37b0efe01 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:55.794365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1028/2017] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:30:55.797501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e779d59-317d-4e0f-96ff-e7fc9695c6e9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:55.807108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e779d59-317d-4e0f-96ff-e7fc9695c6e9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:55.811757 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ef131-d09a-4e43-bde9-38ae8eba071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:55.812449 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ef131-d09a-4e43-bde9-38ae8eba071c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.812847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ef131-d09a-4e43-bde9-38ae8eba071c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.813275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ef131-d09a-4e43-bde9-38ae8eba071c admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.814424 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ef131-d09a-4e43-bde9-38ae8eba071c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:55.818000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ef131-d09a-4e43-bde9-38ae8eba071c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:55.848396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 990/2018] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:55 2022] POST /identity/v3/projects => generated 357 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:30:55.850403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c12c90-4edd-402f-9911-b2e37b0efe01 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:55.870245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c12c90-4edd-402f-9911-b2e37b0efe01 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:55.932623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af7549-40f0-4c44-b4c5-76691265c32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:55.933233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af7549-40f0-4c44-b4c5-76691265c32a admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.933505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af7549-40f0-4c44-b4c5-76691265c32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.933778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af7549-40f0-4c44-b4c5-76691265c32a admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.934417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af7549-40f0-4c44-b4c5-76691265c32a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:55.936396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af7549-40f0-4c44-b4c5-76691265c32a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:55.964137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1029/2019] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:55 2022] POST /identity/v3/users => generated 279 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:55.965378 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ef131-d09a-4e43-bde9-38ae8eba071c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:55.976730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ef131-d09a-4e43-bde9-38ae8eba071c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:55.981595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26dbe2-3d32-408d-a0d2-f2effe11bb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:55.982246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26dbe2-3d32-408d-a0d2-f2effe11bb29 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:55.982710 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26dbe2-3d32-408d-a0d2-f2effe11bb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:55.982979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26dbe2-3d32-408d-a0d2-f2effe11bb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:55.983828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26dbe2-3d32-408d-a0d2-f2effe11bb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:55.985785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26dbe2-3d32-408d-a0d2-f2effe11bb29 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:55.997220 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 991/2020] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:56.001703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af7549-40f0-4c44-b4c5-76691265c32a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:56.012829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af7549-40f0-4c44-b4c5-76691265c32a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:56.019057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd96bc0-083d-442c-a330-3866c8829354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:56.019818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd96bc0-083d-442c-a330-3866c8829354 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:56.020225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd96bc0-083d-442c-a330-3866c8829354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:56.020621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd96bc0-083d-442c-a330-3866c8829354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:56.021363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd96bc0-083d-442c-a330-3866c8829354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:56.023417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd96bc0-083d-442c-a330-3866c8829354 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:56.042151 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1030/2021] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:56.046263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26dbe2-3d32-408d-a0d2-f2effe11bb29 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:56.057940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26dbe2-3d32-408d-a0d2-f2effe11bb29 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:56.063154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f4582-4030-4f69-aaad-6cf05a462d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:56.063839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f4582-4030-4f69-aaad-6cf05a462d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:56.064656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f4582-4030-4f69-aaad-6cf05a462d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:56.065029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f4582-4030-4f69-aaad-6cf05a462d58 admin admin] PATH_INFO: `/v3/projects/b3db6040d48f46daa4960951a27d549c/users/b9f1fc7c871645578658823af37ca106/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:56.082120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f4582-4030-4f69-aaad-6cf05a462d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3db6040d48f46daa4960951a27d549c, user_id=b9f1fc7c871645578658823af37ca106, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:56.084974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f4582-4030-4f69-aaad-6cf05a462d58 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:56.099589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 992/2022] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:56 2022] PUT /identity/v3/projects/b3db6040d48f46daa4960951a27d549c/users/b9f1fc7c871645578658823af37ca106/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:56.109750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ae6f8-ed4f-4121-a949-22c14afbf022 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:56.110144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ae6f8-ed4f-4121-a949-22c14afbf022 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:56.110566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ae6f8-ed4f-4121-a949-22c14afbf022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:56.141957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-dd0ae6f8-ed4f-4121-a949-22c14afbf022 None None] MFA Rules not processed for user `b9f1fc7c871645578658823af37ca106`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:56.143821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dd0ae6f8-ed4f-4121-a949-22c14afbf022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:56.211830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1031/2023] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:56.216210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f4582-4030-4f69-aaad-6cf05a462d58 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:56.277057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f4582-4030-4f69-aaad-6cf05a462d58 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:56.281661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbb925-c2a4-4eba-a8fc-ed77a5a57071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:56.282259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbb925-c2a4-4eba-a8fc-ed77a5a57071 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:56.282545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbb925-c2a4-4eba-a8fc-ed77a5a57071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:56.283337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbb925-c2a4-4eba-a8fc-ed77a5a57071 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:56.284228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbb925-c2a4-4eba-a8fc-ed77a5a57071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:56.286167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbb925-c2a4-4eba-a8fc-ed77a5a57071 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:56.313454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 993/2024] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:56 2022] POST /identity/v3/projects => generated 353 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:30:56.317587 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ae6f8-ed4f-4121-a949-22c14afbf022 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:56.331786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ae6f8-ed4f-4121-a949-22c14afbf022 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:56.335731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:56.393366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57d62f-334b-4c78-87ac-8f6eeb54c60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:56.394019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57d62f-334b-4c78-87ac-8f6eeb54c60b admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:56.394330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57d62f-334b-4c78-87ac-8f6eeb54c60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:56.394601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57d62f-334b-4c78-87ac-8f6eeb54c60b admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:56.395729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57d62f-334b-4c78-87ac-8f6eeb54c60b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:56.397964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57d62f-334b-4c78-87ac-8f6eeb54c60b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:56.420807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1032/2025] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:56 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:30:56.425482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbb925-c2a4-4eba-a8fc-ed77a5a57071 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:56.436170 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbb925-c2a4-4eba-a8fc-ed77a5a57071 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:56.441113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f46cd-0079-4538-b377-49f556dcfb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:56.441711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f46cd-0079-4538-b377-49f556dcfb37 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:56.442001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f46cd-0079-4538-b377-49f556dcfb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:56.442363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f46cd-0079-4538-b377-49f556dcfb37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:56.443066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f46cd-0079-4538-b377-49f556dcfb37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:56.444707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f46cd-0079-4538-b377-49f556dcfb37 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:56.455955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 994/2026] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:56.460216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57d62f-334b-4c78-87ac-8f6eeb54c60b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:56.470003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57d62f-334b-4c78-87ac-8f6eeb54c60b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:56.475188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3102ad1-661d-412d-b8d3-ab5e005a6317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:56.475827 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3102ad1-661d-412d-b8d3-ab5e005a6317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:56.476098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3102ad1-661d-412d-b8d3-ab5e005a6317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:56.476374 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3102ad1-661d-412d-b8d3-ab5e005a6317 admin admin] PATH_INFO: `/v3/projects/a7f57e444a95429796a90c4e2ecec078/users/9584fe50aba94deca8f614568a99027a/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:56.490811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3102ad1-661d-412d-b8d3-ab5e005a6317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f57e444a95429796a90c4e2ecec078, user_id=9584fe50aba94deca8f614568a99027a, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:56.493471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3102ad1-661d-412d-b8d3-ab5e005a6317 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:56.505983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1033/2027] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:56 2022] PUT /identity/v3/projects/a7f57e444a95429796a90c4e2ecec078/users/9584fe50aba94deca8f614568a99027a/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:56.510217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f46cd-0079-4538-b377-49f556dcfb37 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:56.522167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f46cd-0079-4538-b377-49f556dcfb37 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:56.573230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:56.582799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900147b6-2508-4e12-ad5d-8ea181bb848e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:56.583562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900147b6-2508-4e12-ad5d-8ea181bb848e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:56.584045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900147b6-2508-4e12-ad5d-8ea181bb848e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:56.584320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900147b6-2508-4e12-ad5d-8ea181bb848e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:56.584995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900147b6-2508-4e12-ad5d-8ea181bb848e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:56.586615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900147b6-2508-4e12-ad5d-8ea181bb848e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:56.597188 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 995/2028] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:30:56.601477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3102ad1-661d-412d-b8d3-ab5e005a6317 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:56.612253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3102ad1-661d-412d-b8d3-ab5e005a6317 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:56.616791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef85e4b-2ceb-4b1f-b575-2e8d1d0e3f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:56.617431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef85e4b-2ceb-4b1f-b575-2e8d1d0e3f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:56.617707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef85e4b-2ceb-4b1f-b575-2e8d1d0e3f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:56.617977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef85e4b-2ceb-4b1f-b575-2e8d1d0e3f91 admin admin] PATH_INFO: `/v3/projects/a7f57e444a95429796a90c4e2ecec078/users/9584fe50aba94deca8f614568a99027a/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:56.619843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef85e4b-2ceb-4b1f-b575-2e8d1d0e3f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f57e444a95429796a90c4e2ecec078, user_id=9584fe50aba94deca8f614568a99027a, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:56.622472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef85e4b-2ceb-4b1f-b575-2e8d1d0e3f91 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:56.634864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1034/2029] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:56 2022] PUT /identity/v3/projects/a7f57e444a95429796a90c4e2ecec078/users/9584fe50aba94deca8f614568a99027a/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:56.646066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb7967-3465-4634-8afd-1de6db1947f9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:56.646546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb7967-3465-4634-8afd-1de6db1947f9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:56.646915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb7967-3465-4634-8afd-1de6db1947f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:56.678102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6acb7967-3465-4634-8afd-1de6db1947f9 None None] MFA Rules not processed for user `9584fe50aba94deca8f614568a99027a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:56.679338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6acb7967-3465-4634-8afd-1de6db1947f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:56.772547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 996/2030] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:56 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:56.806737 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef85e4b-2ceb-4b1f-b575-2e8d1d0e3f91 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:56.809481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-aef85e4b-2ceb-4b1f-b575-2e8d1d0e3f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:56.878288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef85e4b-2ceb-4b1f-b575-2e8d1d0e3f91 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:56.882130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c193c5-73d2-4f73-bbb0-59140488a8db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:56.882739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c193c5-73d2-4f73-bbb0-59140488a8db service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:56.883018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c193c5-73d2-4f73-bbb0-59140488a8db service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:56.883287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c193c5-73d2-4f73-bbb0-59140488a8db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:56.892933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c193c5-73d2-4f73-bbb0-59140488a8db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:56.894365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c193c5-73d2-4f73-bbb0-59140488a8db service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:56.913022 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1035/2031] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:56 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:57.005743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb7967-3465-4634-8afd-1de6db1947f9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:57.016024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb7967-3465-4634-8afd-1de6db1947f9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:57.019852 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c30a67-3844-4624-82d7-0d0af19d1e88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:57.020479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30a67-3844-4624-82d7-0d0af19d1e88 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:57.020755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30a67-3844-4624-82d7-0d0af19d1e88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:57.021020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30a67-3844-4624-82d7-0d0af19d1e88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:57.083695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c30a67-3844-4624-82d7-0d0af19d1e88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:57.085160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c30a67-3844-4624-82d7-0d0af19d1e88 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:57.093839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 997/2032] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:57.276216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c193c5-73d2-4f73-bbb0-59140488a8db service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:57.277661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-70c193c5-73d2-4f73-bbb0-59140488a8db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:57.342235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c193c5-73d2-4f73-bbb0-59140488a8db service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:57.347452 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:57.348109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:57.348429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:57.348706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] PATH_INFO: `/v3/users/26ad8826b4624ef88de4cf8eaf08f236` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:57.349840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ad8826b4624ef88de4cf8eaf08f236)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:57.351750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:57.354084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:57.400299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ad8826b4624ef88de4cf8eaf08f236'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:57.407110 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ad8826b4624ef88de4cf8eaf08f236'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:57.408405 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ad8826b4624ef88de4cf8eaf08f236'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:57.416493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ad8826b4624ef88de4cf8eaf08f236'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:57.430078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ad8826b4624ef88de4cf8eaf08f236'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:57.436228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1036/2033] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:57 2022] DELETE /identity/v3/users/26ad8826b4624ef88de4cf8eaf08f236 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:57.440850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ac08c-45a3-46be-97b7-e4094ca507ec None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:57.441131 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ac08c-45a3-46be-97b7-e4094ca507ec None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:57.441423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ac08c-45a3-46be-97b7-e4094ca507ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:57.460747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d70ac08c-45a3-46be-97b7-e4094ca507ec None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:57.461714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d70ac08c-45a3-46be-97b7-e4094ca507ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:57.524316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 998/2034] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:57.535518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:57.537199 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:57.600455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd53f5f-6e43-49e6-903b-8ae329722b3d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:57.604067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be72e63-b433-431d-a633-ae892e3beedc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:57.605718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be72e63-b433-431d-a633-ae892e3beedc service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:57.606050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be72e63-b433-431d-a633-ae892e3beedc service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:57.606361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be72e63-b433-431d-a633-ae892e3beedc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:57.615541 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be72e63-b433-431d-a633-ae892e3beedc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:57.617031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be72e63-b433-431d-a633-ae892e3beedc service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:57.626205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1037/2035] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:30:57.901239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ac08c-45a3-46be-97b7-e4094ca507ec None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:57.902687 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d70ac08c-45a3-46be-97b7-e4094ca507ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:57.916224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ac08c-45a3-46be-97b7-e4094ca507ec None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:57.920754 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:57.921392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:57.921653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:57.922361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] PATH_INFO: `/v3/projects/d6663e12901342e79563ae5fb434a8fd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:57.924233 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6663e12901342e79563ae5fb434a8fd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:57.926136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:57.939032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:57.981831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6663e12901342e79563ae5fb434a8fd', 'request_id': 'req-500ffe0d-d572-454e-9b49-656dbf5bba40'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:57.987794 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6663e12901342e79563ae5fb434a8fd', 'request_id': 'req-500ffe0d-d572-454e-9b49-656dbf5bba40'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:57.993920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6663e12901342e79563ae5fb434a8fd', 'request_id': 'req-500ffe0d-d572-454e-9b49-656dbf5bba40'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.006580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] The token cache is being invalidate because project d6663e12901342e79563ae5fb434a8fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:58.007276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.007811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.009963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 999/2036] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:57 2022] DELETE /identity/v3/projects/d6663e12901342e79563ae5fb434a8fd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:58.013981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be72e63-b433-431d-a633-ae892e3beedc service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:58.023883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be72e63-b433-431d-a633-ae892e3beedc service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:58.081852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:58.082477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:58.082749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:58.083014 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] PATH_INFO: `/v3/users/7f0d2ced677a43448e7bd040f6994fdc` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:58.084120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0d2ced677a43448e7bd040f6994fdc)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:58.086043 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:58.113323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:58.133327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0d2ced677a43448e7bd040f6994fdc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.140401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0d2ced677a43448e7bd040f6994fdc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.141206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0d2ced677a43448e7bd040f6994fdc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.149381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0d2ced677a43448e7bd040f6994fdc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.161879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0d2ced677a43448e7bd040f6994fdc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.168131 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1038/2037] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:58 2022] DELETE /identity/v3/users/7f0d2ced677a43448e7bd040f6994fdc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:58.453204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:58.455480 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:58.471583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ffe0d-d572-454e-9b49-656dbf5bba40 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:58.528316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:58.528971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:58.529251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:58.529520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] PATH_INFO: `/v3/projects/6160361f281341edaf52cbcb443b58c1` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:58.530612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6160361f281341edaf52cbcb443b58c1)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:58.532659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:58.578684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6160361f281341edaf52cbcb443b58c1', 'request_id': 'req-11351ea2-dc7a-4bae-95db-a015b66fd7b0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.584805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6160361f281341edaf52cbcb443b58c1', 'request_id': 'req-11351ea2-dc7a-4bae-95db-a015b66fd7b0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.590894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6160361f281341edaf52cbcb443b58c1', 'request_id': 'req-11351ea2-dc7a-4bae-95db-a015b66fd7b0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.603324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] The token cache is being invalidate because project 6160361f281341edaf52cbcb443b58c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:58.603610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.604230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.605759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1000/2038] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:58 2022] DELETE /identity/v3/projects/6160361f281341edaf52cbcb443b58c1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:58.609832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:58.611271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:58.623636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7aa45a-9d3a-4c24-86fa-0f8dc3b0bb56 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:58.681534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bb548-4587-46f6-a19f-1166363fe2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:58.682167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bb548-4587-46f6-a19f-1166363fe2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:58.682441 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bb548-4587-46f6-a19f-1166363fe2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:58.682714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bb548-4587-46f6-a19f-1166363fe2f3 admin admin] PATH_INFO: `/v3/users/e9a36174c9ac4aa3934fbdf073442287` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:58.683837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2bb548-4587-46f6-a19f-1166363fe2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9a36174c9ac4aa3934fbdf073442287)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:58.685796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2bb548-4587-46f6-a19f-1166363fe2f3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:58.737156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9b2bb548-4587-46f6-a19f-1166363fe2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9a36174c9ac4aa3934fbdf073442287'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.745019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9b2bb548-4587-46f6-a19f-1166363fe2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9a36174c9ac4aa3934fbdf073442287'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.745740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9b2bb548-4587-46f6-a19f-1166363fe2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9a36174c9ac4aa3934fbdf073442287'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.753817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9b2bb548-4587-46f6-a19f-1166363fe2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9a36174c9ac4aa3934fbdf073442287'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.766799 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9b2bb548-4587-46f6-a19f-1166363fe2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9a36174c9ac4aa3934fbdf073442287'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:58.773251 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1039/2039] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:30:58 2022] DELETE /identity/v3/users/e9a36174c9ac4aa3934fbdf073442287 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:30:59.040317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:59.041855 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:59.054868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11351ea2-dc7a-4bae-95db-a015b66fd7b0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:59.085246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:59.120961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:59.121724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:59.122020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:59.122296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] PATH_INFO: `/v3/projects/aa419d9d4c604daabaf1e957f1fd5ca4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:59.123578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa419d9d4c604daabaf1e957f1fd5ca4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:59.125559 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:59.174862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa419d9d4c604daabaf1e957f1fd5ca4', 'request_id': 'req-98452ef2-c457-42cf-abfb-4eccde8a24a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:59.180263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa419d9d4c604daabaf1e957f1fd5ca4', 'request_id': 'req-98452ef2-c457-42cf-abfb-4eccde8a24a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:59.186630 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa419d9d4c604daabaf1e957f1fd5ca4', 'request_id': 'req-98452ef2-c457-42cf-abfb-4eccde8a24a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:59.203196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] The token cache is being invalidate because project aa419d9d4c604daabaf1e957f1fd5ca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:30:59.203479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:59.204085 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-98452ef2-c457-42cf-abfb-4eccde8a24a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:30:59.205678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1001/2040] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:30:59 2022] DELETE /identity/v3/projects/aa419d9d4c604daabaf1e957f1fd5ca4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:59.210675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bdf89-2601-4390-afdc-824893d0af0f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:59.210965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bdf89-2601-4390-afdc-824893d0af0f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:59.211237 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bdf89-2601-4390-afdc-824893d0af0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:59.231659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8a3bdf89-2601-4390-afdc-824893d0af0f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:59.232874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8a3bdf89-2601-4390-afdc-824893d0af0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:59.250581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:59.298349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1040/2041] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:30:59.306638 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a8d78-4cbb-4ac9-8a05-24bd3cde77ef None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:59.306924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a8d78-4cbb-4ac9-8a05-24bd3cde77ef None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:59.307253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a8d78-4cbb-4ac9-8a05-24bd3cde77ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:59.327581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-031a8d78-4cbb-4ac9-8a05-24bd3cde77ef None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:59.328807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-031a8d78-4cbb-4ac9-8a05-24bd3cde77ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:59.340987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1002/2042] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:59.356330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bdf89-2601-4390-afdc-824893d0af0f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:59.357422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bdf89-2601-4390-afdc-824893d0af0f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:59.362853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641d26e-34f3-4870-bd6f-31560c022d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:59.363443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641d26e-34f3-4870-bd6f-31560c022d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:59.363715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641d26e-34f3-4870-bd6f-31560c022d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:59.363981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641d26e-34f3-4870-bd6f-31560c022d4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:59.364656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641d26e-34f3-4870-bd6f-31560c022d4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:30:59.364984 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641d26e-34f3-4870-bd6f-31560c022d4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:59.366575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641d26e-34f3-4870-bd6f-31560c022d4a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:59.378397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1041/2043] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:30:59.382474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a8d78-4cbb-4ac9-8a05-24bd3cde77ef None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:59.391721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a8d78-4cbb-4ac9-8a05-24bd3cde77ef None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:59.396303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2c9fb-95c0-4228-bd53-d48b8fe86c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:59.397021 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2c9fb-95c0-4228-bd53-d48b8fe86c9c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:59.397299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2c9fb-95c0-4228-bd53-d48b8fe86c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:59.397568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2c9fb-95c0-4228-bd53-d48b8fe86c9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:59.398205 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2c9fb-95c0-4228-bd53-d48b8fe86c9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:59.400083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2c9fb-95c0-4228-bd53-d48b8fe86c9c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:59.427615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1003/2044] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:30:59 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:59.433008 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641d26e-34f3-4870-bd6f-31560c022d4a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:59.442970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641d26e-34f3-4870-bd6f-31560c022d4a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:59.507805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0b88d-2fc9-4b51-8cae-30a0c5278cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:59.508436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0b88d-2fc9-4b51-8cae-30a0c5278cfb admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:59.508731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0b88d-2fc9-4b51-8cae-30a0c5278cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:59.508976 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0b88d-2fc9-4b51-8cae-30a0c5278cfb admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:59.509586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0b88d-2fc9-4b51-8cae-30a0c5278cfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:59.511455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0b88d-2fc9-4b51-8cae-30a0c5278cfb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:59.537315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1042/2045] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:30:59 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:30:59.538487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2c9fb-95c0-4228-bd53-d48b8fe86c9c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:59.549175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2c9fb-95c0-4228-bd53-d48b8fe86c9c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:59.553767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef9d6f-2671-4fb7-a82f-094ed18ebd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:59.554362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9d6f-2671-4fb7-a82f-094ed18ebd71 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:59.554637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9d6f-2671-4fb7-a82f-094ed18ebd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:59.554914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef9d6f-2671-4fb7-a82f-094ed18ebd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:59.555547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef9d6f-2671-4fb7-a82f-094ed18ebd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:59.557178 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef9d6f-2671-4fb7-a82f-094ed18ebd71 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:59.567242 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1004/2046] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:30:59.571344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0b88d-2fc9-4b51-8cae-30a0c5278cfb admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:59.581653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0b88d-2fc9-4b51-8cae-30a0c5278cfb admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:59.586733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbaa92-e115-4560-b8ab-249ad8a1e187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:59.587334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbaa92-e115-4560-b8ab-249ad8a1e187 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:59.587612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbaa92-e115-4560-b8ab-249ad8a1e187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:59.587966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbaa92-e115-4560-b8ab-249ad8a1e187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:59.588664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbaa92-e115-4560-b8ab-249ad8a1e187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:59.590524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbaa92-e115-4560-b8ab-249ad8a1e187 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:59.605823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1043/2047] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:30:59.607352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef9d6f-2671-4fb7-a82f-094ed18ebd71 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:59.620353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef9d6f-2671-4fb7-a82f-094ed18ebd71 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:59.627655 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9e762-752a-479f-bb43-1a6bb4eeddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:59.628312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9e762-752a-479f-bb43-1a6bb4eeddc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:59.629340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9e762-752a-479f-bb43-1a6bb4eeddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:59.629340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9e762-752a-479f-bb43-1a6bb4eeddc6 admin admin] PATH_INFO: `/v3/projects/08e6fee821e7449b92c3df24de169fbc/users/139dfba82fda43f1a058c60b1a4dc818/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:59.646468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9e762-752a-479f-bb43-1a6bb4eeddc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e6fee821e7449b92c3df24de169fbc, user_id=139dfba82fda43f1a058c60b1a4dc818, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:59.649190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9e762-752a-479f-bb43-1a6bb4eeddc6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:59.664464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1005/2048] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:30:59 2022] PUT /identity/v3/projects/08e6fee821e7449b92c3df24de169fbc/users/139dfba82fda43f1a058c60b1a4dc818/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:30:59.670094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c75a29-ba7d-49f5-baff-7692169277e4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:59.670556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c75a29-ba7d-49f5-baff-7692169277e4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:59.670943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c75a29-ba7d-49f5-baff-7692169277e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:59.692395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-18c75a29-ba7d-49f5-baff-7692169277e4 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:30:59.693841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-18c75a29-ba7d-49f5-baff-7692169277e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:59.769519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1044/2049] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:30:59.779462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9e762-752a-479f-bb43-1a6bb4eeddc6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:30:59.781338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e5b9e762-752a-479f-bb43-1a6bb4eeddc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:30:59.858016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9e762-752a-479f-bb43-1a6bb4eeddc6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:30:59.862412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf229c4f-8344-430c-adc4-00d73a4b1c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:30:59.863164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf229c4f-8344-430c-adc4-00d73a4b1c62 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:30:59.863541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf229c4f-8344-430c-adc4-00d73a4b1c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:30:59.863879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf229c4f-8344-430c-adc4-00d73a4b1c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:30:59.873666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf229c4f-8344-430c-adc4-00d73a4b1c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:30:59.875281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf229c4f-8344-430c-adc4-00d73a4b1c62 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:30:59.880425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:30:59.887244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1006/2050] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:00.586457 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d585e98-e5ce-4fb7-ac36-fabd4919c55f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:00.586915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d585e98-e5ce-4fb7-ac36-fabd4919c55f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:00.587396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d585e98-e5ce-4fb7-ac36-fabd4919c55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:00.620227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6d585e98-e5ce-4fb7-ac36-fabd4919c55f None None] MFA Rules not processed for user `139dfba82fda43f1a058c60b1a4dc818`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:00.621671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6d585e98-e5ce-4fb7-ac36-fabd4919c55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:00.642152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:00.716424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1045/2051] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:00.729181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf229c4f-8344-430c-adc4-00d73a4b1c62 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:00.730666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cf229c4f-8344-430c-adc4-00d73a4b1c62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:00.791485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf229c4f-8344-430c-adc4-00d73a4b1c62 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:00.795352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e83ca5-db21-4541-87b2-2fad9a28e8ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:00.796030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e83ca5-db21-4541-87b2-2fad9a28e8ed service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:00.796435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e83ca5-db21-4541-87b2-2fad9a28e8ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:00.796774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e83ca5-db21-4541-87b2-2fad9a28e8ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:00.805604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e83ca5-db21-4541-87b2-2fad9a28e8ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:00.807930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e83ca5-db21-4541-87b2-2fad9a28e8ed service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:00.818304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1007/2052] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:03.232245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f159ff8-afdb-49bf-8ba0-8b67acbf9f8a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:03.232792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f159ff8-afdb-49bf-8ba0-8b67acbf9f8a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:03.232885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f159ff8-afdb-49bf-8ba0-8b67acbf9f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:03.251461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0f159ff8-afdb-49bf-8ba0-8b67acbf9f8a None None] MFA Rules not processed for user `139dfba82fda43f1a058c60b1a4dc818`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:03.252596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0f159ff8-afdb-49bf-8ba0-8b67acbf9f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:03.259263 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1046/2053] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:10.816850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=27 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:13.246531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=22 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:13.571486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953489f-9aa7-49cb-a259-c71d17e9b076 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:13.571954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953489f-9aa7-49cb-a259-c71d17e9b076 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:13.572142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953489f-9aa7-49cb-a259-c71d17e9b076 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:13.573724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1008/2054] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:31:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:31:13.629856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b952b2c-6efa-4854-84cc-bad790b7638e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:13.630398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b952b2c-6efa-4854-84cc-bad790b7638e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:13.631268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b952b2c-6efa-4854-84cc-bad790b7638e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:13.658191 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6b952b2c-6efa-4854-84cc-bad790b7638e None None] MFA Rules not processed for user `0a50fd417ae94859bb713b8b2066a9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:13.660779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6b952b2c-6efa-4854-84cc-bad790b7638e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:13.750791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1047/2055] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:31:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:13.784752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953489f-9aa7-49cb-a259-c71d17e9b076 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:13.786541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3953489f-9aa7-49cb-a259-c71d17e9b076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:13.803798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953489f-9aa7-49cb-a259-c71d17e9b076 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:13.809457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8696825-046f-4853-a946-4fb2b7b9faad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:13.810060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696825-046f-4853-a946-4fb2b7b9faad service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:13.810450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696825-046f-4853-a946-4fb2b7b9faad service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:13.810726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696825-046f-4853-a946-4fb2b7b9faad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:13.819453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696825-046f-4853-a946-4fb2b7b9faad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:13.820997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696825-046f-4853-a946-4fb2b7b9faad service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:13.831548 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1009/2056] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:14.262967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b952b2c-6efa-4854-84cc-bad790b7638e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:14.264477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6b952b2c-6efa-4854-84cc-bad790b7638e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:14.277983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b952b2c-6efa-4854-84cc-bad790b7638e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:14.283828 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:14.284761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:14.285035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:14.285306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] PATH_INFO: `/v3/users/05dfecc5811845e4922d0e47db644552` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:14.286636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05dfecc5811845e4922d0e47db644552)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:14.288854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:14.337308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05dfecc5811845e4922d0e47db644552'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.344058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05dfecc5811845e4922d0e47db644552'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.344760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05dfecc5811845e4922d0e47db644552'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.352048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05dfecc5811845e4922d0e47db644552'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.363888 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05dfecc5811845e4922d0e47db644552'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.370901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1048/2057] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:14 2022] DELETE /identity/v3/users/05dfecc5811845e4922d0e47db644552 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:31:14.558985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8696825-046f-4853-a946-4fb2b7b9faad service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:14.570662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8696825-046f-4853-a946-4fb2b7b9faad service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:14.630279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:14.630953 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:14.631238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:14.631505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] PATH_INFO: `/v3/projects/04e31b183c914d77a8bcc34b214ffa8b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:14.632635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e31b183c914d77a8bcc34b214ffa8b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:14.634609 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:14.681249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e31b183c914d77a8bcc34b214ffa8b', 'request_id': 'req-2ba25d1a-d6aa-4319-af39-97563783d389'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.686800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e31b183c914d77a8bcc34b214ffa8b', 'request_id': 'req-2ba25d1a-d6aa-4319-af39-97563783d389'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.693060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e31b183c914d77a8bcc34b214ffa8b', 'request_id': 'req-2ba25d1a-d6aa-4319-af39-97563783d389'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.706064 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] The token cache is being invalidate because project 04e31b183c914d77a8bcc34b214ffa8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:14.706353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.706994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.708509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1010/2058] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:14 2022] DELETE /identity/v3/projects/04e31b183c914d77a8bcc34b214ffa8b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:14.712736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:14.714166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:14.730974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811d17f-8893-4d68-8fe1-39120ed0d2e3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:14.801721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c0b9e-f021-45af-8384-170b38801231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:14.802383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c0b9e-f021-45af-8384-170b38801231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:14.802624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c0b9e-f021-45af-8384-170b38801231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:14.802895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c0b9e-f021-45af-8384-170b38801231 admin admin] PATH_INFO: `/v3/users/85b1c840a9ae4641a38c7c29475ca1e8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:14.804199 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c0b9e-f021-45af-8384-170b38801231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b1c840a9ae4641a38c7c29475ca1e8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:14.806219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c0b9e-f021-45af-8384-170b38801231 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:14.817911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:14.858463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0b1c0b9e-f021-45af-8384-170b38801231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b1c840a9ae4641a38c7c29475ca1e8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.866552 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0b1c0b9e-f021-45af-8384-170b38801231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b1c840a9ae4641a38c7c29475ca1e8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.867355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0b1c0b9e-f021-45af-8384-170b38801231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b1c840a9ae4641a38c7c29475ca1e8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.874914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0b1c0b9e-f021-45af-8384-170b38801231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b1c840a9ae4641a38c7c29475ca1e8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.888496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0b1c0b9e-f021-45af-8384-170b38801231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b1c840a9ae4641a38c7c29475ca1e8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:14.895920 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1049/2059] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:14 2022] DELETE /identity/v3/users/85b1c840a9ae4641a38c7c29475ca1e8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:31:15.070197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:15.072020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:15.085739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba25d1a-d6aa-4319-af39-97563783d389 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:15.154024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:15.154664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:15.154994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:15.155268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] PATH_INFO: `/v3/projects/3f6a23fdbc5c4786be3c1b8db7aee897` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:15.156432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f6a23fdbc5c4786be3c1b8db7aee897)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:15.158924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:15.181708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:15.208362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f6a23fdbc5c4786be3c1b8db7aee897', 'request_id': 'req-4858257d-7be8-4c73-8806-a8ddadc5a5c2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:15.213704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f6a23fdbc5c4786be3c1b8db7aee897', 'request_id': 'req-4858257d-7be8-4c73-8806-a8ddadc5a5c2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:15.219637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f6a23fdbc5c4786be3c1b8db7aee897', 'request_id': 'req-4858257d-7be8-4c73-8806-a8ddadc5a5c2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:15.232195 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] The token cache is being invalidate because project 3f6a23fdbc5c4786be3c1b8db7aee897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:15.232510 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:15.233121 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-4858257d-7be8-4c73-8806-a8ddadc5a5c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:15.234626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1011/2060] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:15 2022] DELETE /identity/v3/projects/3f6a23fdbc5c4786be3c1b8db7aee897 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:31:15.240463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9df71b-2a63-4e91-816c-593e073a1341 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:15.240746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9df71b-2a63-4e91-816c-593e073a1341 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:15.241019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9df71b-2a63-4e91-816c-593e073a1341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:15.259964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5d9df71b-2a63-4e91-816c-593e073a1341 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:15.261173 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5d9df71b-2a63-4e91-816c-593e073a1341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:15.318630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1050/2061] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:15.326821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1ef86-9ee4-4e9c-a6f2-73542784d311 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:15.326997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1ef86-9ee4-4e9c-a6f2-73542784d311 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:15.327274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1ef86-9ee4-4e9c-a6f2-73542784d311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:15.347441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7cc1ef86-9ee4-4e9c-a6f2-73542784d311 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:15.348678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7cc1ef86-9ee4-4e9c-a6f2-73542784d311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:15.356821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1012/2062] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:15.360916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9df71b-2a63-4e91-816c-593e073a1341 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:15.371303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9df71b-2a63-4e91-816c-593e073a1341 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:15.375826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c63c12-b4a5-4515-b27a-89ee74164c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:15.376451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c63c12-b4a5-4515-b27a-89ee74164c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:15.376727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c63c12-b4a5-4515-b27a-89ee74164c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:15.376993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c63c12-b4a5-4515-b27a-89ee74164c36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:15.377637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c63c12-b4a5-4515-b27a-89ee74164c36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:31:15.377939 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c63c12-b4a5-4515-b27a-89ee74164c36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:15.379508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c63c12-b4a5-4515-b27a-89ee74164c36 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:15.395256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1051/2063] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:31:15.399909 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1ef86-9ee4-4e9c-a6f2-73542784d311 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:15.411466 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1ef86-9ee4-4e9c-a6f2-73542784d311 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:15.416781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde043eb-a9e0-4b83-916a-fef7e8ab1c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:15.416905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde043eb-a9e0-4b83-916a-fef7e8ab1c42 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:15.417139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde043eb-a9e0-4b83-916a-fef7e8ab1c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:15.417405 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde043eb-a9e0-4b83-916a-fef7e8ab1c42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:15.418463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde043eb-a9e0-4b83-916a-fef7e8ab1c42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:15.420347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde043eb-a9e0-4b83-916a-fef7e8ab1c42 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:15.453281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1013/2064] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:15 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:15.455157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c63c12-b4a5-4515-b27a-89ee74164c36 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:15.465487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c63c12-b4a5-4515-b27a-89ee74164c36 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:15.526433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f1412-b83f-4ccd-a8bd-eee71fa21dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:15.527035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f1412-b83f-4ccd-a8bd-eee71fa21dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:15.527315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f1412-b83f-4ccd-a8bd-eee71fa21dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:15.527597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f1412-b83f-4ccd-a8bd-eee71fa21dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:15.528217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f1412-b83f-4ccd-a8bd-eee71fa21dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:15.530810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f1412-b83f-4ccd-a8bd-eee71fa21dd7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:15.557037 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1052/2065] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:15 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:15.561394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde043eb-a9e0-4b83-916a-fef7e8ab1c42 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:15.577236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde043eb-a9e0-4b83-916a-fef7e8ab1c42 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:15.584173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b27487-db9f-4153-a088-4c3140405215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:15.584868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b27487-db9f-4153-a088-4c3140405215 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:15.585142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b27487-db9f-4153-a088-4c3140405215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:15.585411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b27487-db9f-4153-a088-4c3140405215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:15.586049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b27487-db9f-4153-a088-4c3140405215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:15.587888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b27487-db9f-4153-a088-4c3140405215 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:15.603251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1014/2066] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:31:15.605706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f1412-b83f-4ccd-a8bd-eee71fa21dd7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:15.614609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:15.622226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f1412-b83f-4ccd-a8bd-eee71fa21dd7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:15.629930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e932e4-436b-4642-99f9-6f63589a3f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:15.630221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e932e4-436b-4642-99f9-6f63589a3f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:15.630493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e932e4-436b-4642-99f9-6f63589a3f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:15.630764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e932e4-436b-4642-99f9-6f63589a3f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:15.631480 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e932e4-436b-4642-99f9-6f63589a3f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:15.633189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e932e4-436b-4642-99f9-6f63589a3f25 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:15.650030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1053/2067] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:15.654310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b27487-db9f-4153-a088-4c3140405215 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:15.665753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b27487-db9f-4153-a088-4c3140405215 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:15.670504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac7981-1db5-40b0-91b2-c3bc4ee9f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:15.671286 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac7981-1db5-40b0-91b2-c3bc4ee9f28d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:15.671569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac7981-1db5-40b0-91b2-c3bc4ee9f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:15.671844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac7981-1db5-40b0-91b2-c3bc4ee9f28d admin admin] PATH_INFO: `/v3/projects/8966e68dee3a4f73b1d4d4d8c748ecec/users/0f4f9dce871844d0aa83ba2a2ba88274/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:15.689044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac7981-1db5-40b0-91b2-c3bc4ee9f28d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8966e68dee3a4f73b1d4d4d8c748ecec, user_id=0f4f9dce871844d0aa83ba2a2ba88274, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:15.692403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac7981-1db5-40b0-91b2-c3bc4ee9f28d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:15.711462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1015/2068] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:15 2022] PUT /identity/v3/projects/8966e68dee3a4f73b1d4d4d8c748ecec/users/0f4f9dce871844d0aa83ba2a2ba88274/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:15.723254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0a2fb-e749-49d5-b1eb-6004b94787a4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:15.723419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0a2fb-e749-49d5-b1eb-6004b94787a4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:15.723584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0a2fb-e749-49d5-b1eb-6004b94787a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:15.756172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ebe0a2fb-e749-49d5-b1eb-6004b94787a4 None None] MFA Rules not processed for user `0f4f9dce871844d0aa83ba2a2ba88274`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:15.757717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ebe0a2fb-e749-49d5-b1eb-6004b94787a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:15.830893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1054/2069] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:15.852632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac7981-1db5-40b0-91b2-c3bc4ee9f28d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:15.855044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ddac7981-1db5-40b0-91b2-c3bc4ee9f28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:15.925141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac7981-1db5-40b0-91b2-c3bc4ee9f28d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:15.929022 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b2cd1-01ab-486b-9670-f73b407acd3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:15.929637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b2cd1-01ab-486b-9670-f73b407acd3f service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:15.929969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b2cd1-01ab-486b-9670-f73b407acd3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:15.930269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b2cd1-01ab-486b-9670-f73b407acd3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:15.939853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b2cd1-01ab-486b-9670-f73b407acd3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:15.941332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b2cd1-01ab-486b-9670-f73b407acd3f service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:15.951620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1016/2070] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:16.492759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0a2fb-e749-49d5-b1eb-6004b94787a4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:16.494276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ebe0a2fb-e749-49d5-b1eb-6004b94787a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:16.568791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0a2fb-e749-49d5-b1eb-6004b94787a4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:16.575242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:16.576130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:16.576517 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:16.577067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] PATH_INFO: `/v3/users/b9f1fc7c871645578658823af37ca106` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:16.578644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f1fc7c871645578658823af37ca106)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:16.581715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:16.639849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f1fc7c871645578658823af37ca106'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:16.650418 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f1fc7c871645578658823af37ca106'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:16.651637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f1fc7c871645578658823af37ca106'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:16.660818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f1fc7c871645578658823af37ca106'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:16.675936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f1fc7c871645578658823af37ca106'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:16.693383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1055/2071] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:16 2022] DELETE /identity/v3/users/b9f1fc7c871645578658823af37ca106 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:31:16.693983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d0314-c9fe-4174-84f5-f0891a4f1f4e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:16.693983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d0314-c9fe-4174-84f5-f0891a4f1f4e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:16.693983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d0314-c9fe-4174-84f5-f0891a4f1f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:16.713859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-084d0314-c9fe-4174-84f5-f0891a4f1f4e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:16.715099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-084d0314-c9fe-4174-84f5-f0891a4f1f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:16.761701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:16.799136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1017/2072] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:16.809956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:16.811488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:16.822089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:16.891093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944efa40-5d15-493f-b789-92addc1bc7da admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:16.897744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e3493-ea3d-444c-b098-c9f7b10e2699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:16.898887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e3493-ea3d-444c-b098-c9f7b10e2699 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:16.899487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e3493-ea3d-444c-b098-c9f7b10e2699 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:16.899891 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e3493-ea3d-444c-b098-c9f7b10e2699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:16.912561 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e3493-ea3d-444c-b098-c9f7b10e2699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:16.914607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e3493-ea3d-444c-b098-c9f7b10e2699 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:16.925874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1056/2073] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:17.208439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d0314-c9fe-4174-84f5-f0891a4f1f4e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:17.209867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-084d0314-c9fe-4174-84f5-f0891a4f1f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:17.224308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d0314-c9fe-4174-84f5-f0891a4f1f4e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:17.230377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:17.231131 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:17.231448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:17.231675 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] PATH_INFO: `/v3/projects/b3db6040d48f46daa4960951a27d549c` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:17.232822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3db6040d48f46daa4960951a27d549c)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:17.234768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:17.283922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3db6040d48f46daa4960951a27d549c', 'request_id': 'req-d004f269-cf14-492a-b30a-6fb11d7fcc53'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.291009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3db6040d48f46daa4960951a27d549c', 'request_id': 'req-d004f269-cf14-492a-b30a-6fb11d7fcc53'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.297600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3db6040d48f46daa4960951a27d549c', 'request_id': 'req-d004f269-cf14-492a-b30a-6fb11d7fcc53'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.313023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] The token cache is being invalidate because project b3db6040d48f46daa4960951a27d549c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:17.313924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.314681 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.316289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1018/2074] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:17 2022] DELETE /identity/v3/projects/b3db6040d48f46daa4960951a27d549c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:17.320468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e3493-ea3d-444c-b098-c9f7b10e2699 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:17.334671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e3493-ea3d-444c-b098-c9f7b10e2699 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:17.396946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013acec-7bf4-414a-b5b3-0b57200e47a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:17.397640 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013acec-7bf4-414a-b5b3-0b57200e47a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:17.397910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013acec-7bf4-414a-b5b3-0b57200e47a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:17.398175 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013acec-7bf4-414a-b5b3-0b57200e47a1 admin admin] PATH_INFO: `/v3/users/9584fe50aba94deca8f614568a99027a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:17.399296 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013acec-7bf4-414a-b5b3-0b57200e47a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9584fe50aba94deca8f614568a99027a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:17.401236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013acec-7bf4-414a-b5b3-0b57200e47a1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:17.449354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c013acec-7bf4-414a-b5b3-0b57200e47a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9584fe50aba94deca8f614568a99027a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.457182 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c013acec-7bf4-414a-b5b3-0b57200e47a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9584fe50aba94deca8f614568a99027a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.457857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c013acec-7bf4-414a-b5b3-0b57200e47a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9584fe50aba94deca8f614568a99027a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.465810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c013acec-7bf4-414a-b5b3-0b57200e47a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9584fe50aba94deca8f614568a99027a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.479103 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c013acec-7bf4-414a-b5b3-0b57200e47a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9584fe50aba94deca8f614568a99027a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.485464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1057/2075] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:17 2022] DELETE /identity/v3/users/9584fe50aba94deca8f614568a99027a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:17.783270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:17.786270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:17.807116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004f269-cf14-492a-b30a-6fb11d7fcc53 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:17.896767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:17.897403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:17.897677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:17.897942 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] PATH_INFO: `/v3/projects/a7f57e444a95429796a90c4e2ecec078` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:17.899037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7f57e444a95429796a90c4e2ecec078)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:17.900971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:17.947285 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7f57e444a95429796a90c4e2ecec078', 'request_id': 'req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.953388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7f57e444a95429796a90c4e2ecec078', 'request_id': 'req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.959590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7f57e444a95429796a90c4e2ecec078', 'request_id': 'req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.972393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:17.976887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] The token cache is being invalidate because project a7f57e444a95429796a90c4e2ecec078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:17.977540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.978062 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-37c6d69e-95e8-4cd7-a100-e9fd2d8f5457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:17.979876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1019/2076] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:17 2022] DELETE /identity/v3/projects/a7f57e444a95429796a90c4e2ecec078 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:17.984981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e80b0a-1b0b-486e-9c9c-407119c3db5b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:17.986117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e80b0a-1b0b-486e-9c9c-407119c3db5b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:17.986117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e80b0a-1b0b-486e-9c9c-407119c3db5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.006884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-06e80b0a-1b0b-486e-9c9c-407119c3db5b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:18.008108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-06e80b0a-1b0b-486e-9c9c-407119c3db5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:18.028087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:18.073898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1058/2077] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:18.082145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf567f0-8c5b-45fc-a8c8-4f83339d2eea None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.082428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf567f0-8c5b-45fc-a8c8-4f83339d2eea None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.082703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf567f0-8c5b-45fc-a8c8-4f83339d2eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.104819 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ecf567f0-8c5b-45fc-a8c8-4f83339d2eea None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:18.106027 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ecf567f0-8c5b-45fc-a8c8-4f83339d2eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:18.114745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1020/2078] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:18.119153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e80b0a-1b0b-486e-9c9c-407119c3db5b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.128844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e80b0a-1b0b-486e-9c9c-407119c3db5b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.133357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2aa40-d340-4d27-9e34-b464b70a1cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.133952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2aa40-d340-4d27-9e34-b464b70a1cfb admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.134228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2aa40-d340-4d27-9e34-b464b70a1cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.134500 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2aa40-d340-4d27-9e34-b464b70a1cfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.135142 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2aa40-d340-4d27-9e34-b464b70a1cfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:31:18.135439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2aa40-d340-4d27-9e34-b464b70a1cfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:18.137052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2aa40-d340-4d27-9e34-b464b70a1cfb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:18.154272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1059/2079] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:31:18.155400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf567f0-8c5b-45fc-a8c8-4f83339d2eea None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.165692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf567f0-8c5b-45fc-a8c8-4f83339d2eea None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.170426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb7ec4-dc64-407e-8540-0bc77b599cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.171031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb7ec4-dc64-407e-8540-0bc77b599cbd admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.171310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb7ec4-dc64-407e-8540-0bc77b599cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.171581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb7ec4-dc64-407e-8540-0bc77b599cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.172234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb7ec4-dc64-407e-8540-0bc77b599cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:18.174793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb7ec4-dc64-407e-8540-0bc77b599cbd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:18.215660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2aa40-d340-4d27-9e34-b464b70a1cfb admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.216226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1021/2080] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:18 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:31:18.228276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2aa40-d340-4d27-9e34-b464b70a1cfb admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.298133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c563fe-325a-4e65-bbe4-78757fae6db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.298765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c563fe-325a-4e65-bbe4-78757fae6db5 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.299071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c563fe-325a-4e65-bbe4-78757fae6db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.299527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c563fe-325a-4e65-bbe4-78757fae6db5 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.300464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c563fe-325a-4e65-bbe4-78757fae6db5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:18.302829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c563fe-325a-4e65-bbe4-78757fae6db5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:18.328162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1060/2081] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:18 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:18.329990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb7ec4-dc64-407e-8540-0bc77b599cbd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.340505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb7ec4-dc64-407e-8540-0bc77b599cbd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.345848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ec80a-8a9b-45e3-9111-971d287fda1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.346550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ec80a-8a9b-45e3-9111-971d287fda1d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.346816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ec80a-8a9b-45e3-9111-971d287fda1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.347122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ec80a-8a9b-45e3-9111-971d287fda1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.347783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ec80a-8a9b-45e3-9111-971d287fda1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:18.349460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ec80a-8a9b-45e3-9111-971d287fda1d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:18.366101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c563fe-325a-4e65-bbe4-78757fae6db5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.366854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1022/2082] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:31:18.376992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c563fe-325a-4e65-bbe4-78757fae6db5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.381986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927ca04-7be7-4239-a350-e4acc3beb454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.382716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927ca04-7be7-4239-a350-e4acc3beb454 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.383167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927ca04-7be7-4239-a350-e4acc3beb454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.383460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927ca04-7be7-4239-a350-e4acc3beb454 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.384208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927ca04-7be7-4239-a350-e4acc3beb454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:18.386214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927ca04-7be7-4239-a350-e4acc3beb454 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:18.407265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1061/2083] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:18.413734 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ec80a-8a9b-45e3-9111-971d287fda1d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.428221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ec80a-8a9b-45e3-9111-971d287fda1d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.433004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e0ce1-de4a-45fb-a150-be95983b0731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.433791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e0ce1-de4a-45fb-a150-be95983b0731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.434407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e0ce1-de4a-45fb-a150-be95983b0731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.434677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e0ce1-de4a-45fb-a150-be95983b0731 admin admin] PATH_INFO: `/v3/projects/608212b6cf0948e284d0d5c0f1bc21a7/users/b5683dea23564c6599d7e7995399b32e/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.451556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e0ce1-de4a-45fb-a150-be95983b0731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=608212b6cf0948e284d0d5c0f1bc21a7, user_id=b5683dea23564c6599d7e7995399b32e, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:18.454833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e0ce1-de4a-45fb-a150-be95983b0731 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:18.469414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1023/2084] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:18 2022] PUT /identity/v3/projects/608212b6cf0948e284d0d5c0f1bc21a7/users/b5683dea23564c6599d7e7995399b32e/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:31:18.480686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6703c-f544-436e-91ad-8fc8c5e4cfbc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.480969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6703c-f544-436e-91ad-8fc8c5e4cfbc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.481307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6703c-f544-436e-91ad-8fc8c5e4cfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.514624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bbe6703c-f544-436e-91ad-8fc8c5e4cfbc None None] MFA Rules not processed for user `b5683dea23564c6599d7e7995399b32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:18.516165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bbe6703c-f544-436e-91ad-8fc8c5e4cfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:18.589313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1062/2085] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:18.593204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e0ce1-de4a-45fb-a150-be95983b0731 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.660994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e0ce1-de4a-45fb-a150-be95983b0731 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.665573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a2ece-91e0-4d54-a0ae-450d21f9bfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.666178 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a2ece-91e0-4d54-a0ae-450d21f9bfa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.666457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a2ece-91e0-4d54-a0ae-450d21f9bfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.666726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a2ece-91e0-4d54-a0ae-450d21f9bfa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.667361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a2ece-91e0-4d54-a0ae-450d21f9bfa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:18.669282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a2ece-91e0-4d54-a0ae-450d21f9bfa4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:18.695667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1024/2086] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:18 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:18.699727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6703c-f544-436e-91ad-8fc8c5e4cfbc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.710386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6703c-f544-436e-91ad-8fc8c5e4cfbc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.729508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:18.770279 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1cb37-0276-4e5a-927a-656a36296f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.770887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1cb37-0276-4e5a-927a-656a36296f6a admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.771164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1cb37-0276-4e5a-927a-656a36296f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.771464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1cb37-0276-4e5a-927a-656a36296f6a admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.772067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1cb37-0276-4e5a-927a-656a36296f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:18.774020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1cb37-0276-4e5a-927a-656a36296f6a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:18.795486 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1063/2087] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:18 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:31:18.800072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a2ece-91e0-4d54-a0ae-450d21f9bfa4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.810474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a2ece-91e0-4d54-a0ae-450d21f9bfa4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.815094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e595f-39fe-4296-811f-4cd3cafabbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.815677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e595f-39fe-4296-811f-4cd3cafabbfb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.815954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e595f-39fe-4296-811f-4cd3cafabbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.816225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e595f-39fe-4296-811f-4cd3cafabbfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.816907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e595f-39fe-4296-811f-4cd3cafabbfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:18.818489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e595f-39fe-4296-811f-4cd3cafabbfb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:18.828779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1025/2088] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:18.832845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1cb37-0276-4e5a-927a-656a36296f6a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.842518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1cb37-0276-4e5a-927a-656a36296f6a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.847822 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb1ed4-449f-46b4-8f1f-e48364883ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.848496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb1ed4-449f-46b4-8f1f-e48364883ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.848774 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb1ed4-449f-46b4-8f1f-e48364883ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.849045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb1ed4-449f-46b4-8f1f-e48364883ab7 admin admin] PATH_INFO: `/v3/projects/ce2b0e8728aa44adbc69448b73d0c99a/users/8796e444c4084482b7defa130dc72284/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.864139 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb1ed4-449f-46b4-8f1f-e48364883ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2b0e8728aa44adbc69448b73d0c99a, user_id=8796e444c4084482b7defa130dc72284, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:18.866792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb1ed4-449f-46b4-8f1f-e48364883ab7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:18.879185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1064/2089] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:18 2022] PUT /identity/v3/projects/ce2b0e8728aa44adbc69448b73d0c99a/users/8796e444c4084482b7defa130dc72284/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:18.883225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e595f-39fe-4296-811f-4cd3cafabbfb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.893082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e595f-39fe-4296-811f-4cd3cafabbfb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.953648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b8023-4199-4685-97c8-35cf8df89c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.954914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b8023-4199-4685-97c8-35cf8df89c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.955251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b8023-4199-4685-97c8-35cf8df89c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.955585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b8023-4199-4685-97c8-35cf8df89c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.956446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b8023-4199-4685-97c8-35cf8df89c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:18.958097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b8023-4199-4685-97c8-35cf8df89c61 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:18.974316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1026/2090] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:18.979231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb1ed4-449f-46b4-8f1f-e48364883ab7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:18.990936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb1ed4-449f-46b4-8f1f-e48364883ab7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:18.995814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01d8cc-7098-4ccb-87bc-5a0308953cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:18.996506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01d8cc-7098-4ccb-87bc-5a0308953cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:18.996782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01d8cc-7098-4ccb-87bc-5a0308953cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:18.997053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01d8cc-7098-4ccb-87bc-5a0308953cd6 admin admin] PATH_INFO: `/v3/projects/ce2b0e8728aa44adbc69448b73d0c99a/users/8796e444c4084482b7defa130dc72284/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:18.999029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01d8cc-7098-4ccb-87bc-5a0308953cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2b0e8728aa44adbc69448b73d0c99a, user_id=8796e444c4084482b7defa130dc72284, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:19.001769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01d8cc-7098-4ccb-87bc-5a0308953cd6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:19.013807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1065/2091] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:18 2022] PUT /identity/v3/projects/ce2b0e8728aa44adbc69448b73d0c99a/users/8796e444c4084482b7defa130dc72284/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:31:19.029344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cffb5-a6f7-4a66-9982-3a5af2ef738b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:19.029630 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cffb5-a6f7-4a66-9982-3a5af2ef738b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:19.029900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cffb5-a6f7-4a66-9982-3a5af2ef738b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:19.045041 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:19.060709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5e9cffb5-a6f7-4a66-9982-3a5af2ef738b None None] MFA Rules not processed for user `8796e444c4084482b7defa130dc72284`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:19.061901 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5e9cffb5-a6f7-4a66-9982-3a5af2ef738b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:19.128256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1027/2092] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:19.139065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01d8cc-7098-4ccb-87bc-5a0308953cd6 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:19.140776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ab01d8cc-7098-4ccb-87bc-5a0308953cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:19.198211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01d8cc-7098-4ccb-87bc-5a0308953cd6 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:19.201804 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a3daf-c18a-490a-9150-2d1ee4c20c66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:19.202391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a3daf-c18a-490a-9150-2d1ee4c20c66 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:19.202665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a3daf-c18a-490a-9150-2d1ee4c20c66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:19.202929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a3daf-c18a-490a-9150-2d1ee4c20c66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:19.212448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a3daf-c18a-490a-9150-2d1ee4c20c66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:19.213852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a3daf-c18a-490a-9150-2d1ee4c20c66 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:19.223507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1066/2093] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:20.592283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cffb5-a6f7-4a66-9982-3a5af2ef738b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:20.652461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cffb5-a6f7-4a66-9982-3a5af2ef738b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:20.657306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:20.657983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:20.658198 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:20.658469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] PATH_INFO: `/v3/users/b5683dea23564c6599d7e7995399b32e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:20.659839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5683dea23564c6599d7e7995399b32e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:20.661818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:20.707427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5683dea23564c6599d7e7995399b32e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:20.714193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5683dea23564c6599d7e7995399b32e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:20.714953 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5683dea23564c6599d7e7995399b32e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:20.722799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5683dea23564c6599d7e7995399b32e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:20.734516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5683dea23564c6599d7e7995399b32e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:20.740756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1028/2094] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:20 2022] DELETE /identity/v3/users/b5683dea23564c6599d7e7995399b32e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:20.745328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16583b-9fcf-427d-8c36-8157e8688b44 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:20.745767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16583b-9fcf-427d-8c36-8157e8688b44 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:20.745898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16583b-9fcf-427d-8c36-8157e8688b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:20.765092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bd16583b-9fcf-427d-8c36-8157e8688b44 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:20.766265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bd16583b-9fcf-427d-8c36-8157e8688b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:20.824362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1067/2095] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:20.833934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:20.835328 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:20.849072 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:20.893027 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dff6c-8896-4587-840c-07de091a050f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:20.896656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c2a33-dc25-4674-aa48-f64becf44d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:20.897254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c2a33-dc25-4674-aa48-f64becf44d74 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:20.897531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c2a33-dc25-4674-aa48-f64becf44d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:20.897802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c2a33-dc25-4674-aa48-f64becf44d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:20.907997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c2a33-dc25-4674-aa48-f64becf44d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:20.909900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c2a33-dc25-4674-aa48-f64becf44d74 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:20.919635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1029/2096] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:21.183337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16583b-9fcf-427d-8c36-8157e8688b44 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:21.184795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bd16583b-9fcf-427d-8c36-8157e8688b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:21.197019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16583b-9fcf-427d-8c36-8157e8688b44 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:21.201608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:21.203483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:21.203769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:21.204561 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] PATH_INFO: `/v3/projects/608212b6cf0948e284d0d5c0f1bc21a7` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:21.205699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=608212b6cf0948e284d0d5c0f1bc21a7)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:21.208608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:21.222777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:21.260108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '608212b6cf0948e284d0d5c0f1bc21a7', 'request_id': 'req-e51aa85f-21b3-4370-b90e-d41a4d7966a2'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.265820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '608212b6cf0948e284d0d5c0f1bc21a7', 'request_id': 'req-e51aa85f-21b3-4370-b90e-d41a4d7966a2'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.272722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '608212b6cf0948e284d0d5c0f1bc21a7', 'request_id': 'req-e51aa85f-21b3-4370-b90e-d41a4d7966a2'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.288455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] The token cache is being invalidate because project 608212b6cf0948e284d0d5c0f1bc21a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:21.288745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.289389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.290896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1068/2097] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:21 2022] DELETE /identity/v3/projects/608212b6cf0948e284d0d5c0f1bc21a7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:21.294869 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c2a33-dc25-4674-aa48-f64becf44d74 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:21.305105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c2a33-dc25-4674-aa48-f64becf44d74 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:21.366050 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac6353-68ca-49ed-b166-529d19415d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:21.366677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6353-68ca-49ed-b166-529d19415d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:21.367032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6353-68ca-49ed-b166-529d19415d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:21.367364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6353-68ca-49ed-b166-529d19415d19 admin admin] PATH_INFO: `/v3/users/8796e444c4084482b7defa130dc72284` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:21.368547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac6353-68ca-49ed-b166-529d19415d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8796e444c4084482b7defa130dc72284)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:21.370447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac6353-68ca-49ed-b166-529d19415d19 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:21.428837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-25ac6353-68ca-49ed-b166-529d19415d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8796e444c4084482b7defa130dc72284'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.437638 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-25ac6353-68ca-49ed-b166-529d19415d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8796e444c4084482b7defa130dc72284'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.438449 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-25ac6353-68ca-49ed-b166-529d19415d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8796e444c4084482b7defa130dc72284'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.446232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-25ac6353-68ca-49ed-b166-529d19415d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8796e444c4084482b7defa130dc72284'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.459287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-25ac6353-68ca-49ed-b166-529d19415d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8796e444c4084482b7defa130dc72284'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.465132 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1030/2098] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:21 2022] DELETE /identity/v3/users/8796e444c4084482b7defa130dc72284 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:21.732261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:21.733719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:21.747572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aa85f-21b3-4370-b90e-d41a4d7966a2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:21.804891 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:21.805519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:21.805791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:21.806057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] PATH_INFO: `/v3/projects/ce2b0e8728aa44adbc69448b73d0c99a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:21.807150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2b0e8728aa44adbc69448b73d0c99a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:21.809175 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:21.856052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2b0e8728aa44adbc69448b73d0c99a', 'request_id': 'req-2e469824-c692-4b67-9632-f79d8d5d97f9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.861400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2b0e8728aa44adbc69448b73d0c99a', 'request_id': 'req-2e469824-c692-4b67-9632-f79d8d5d97f9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.867470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2b0e8728aa44adbc69448b73d0c99a', 'request_id': 'req-2e469824-c692-4b67-9632-f79d8d5d97f9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.879967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] The token cache is being invalidate because project ce2b0e8728aa44adbc69448b73d0c99a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:21.880258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.880902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2e469824-c692-4b67-9632-f79d8d5d97f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:21.882573 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1069/2099] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:21 2022] DELETE /identity/v3/projects/ce2b0e8728aa44adbc69448b73d0c99a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:31:21.887376 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d38cda-e568-483a-a860-bd380c55eec4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:21.887868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d38cda-e568-483a-a860-bd380c55eec4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:21.887961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d38cda-e568-483a-a860-bd380c55eec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:21.909275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-98d38cda-e568-483a-a860-bd380c55eec4 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:21.910473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-98d38cda-e568-483a-a860-bd380c55eec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:21.939740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:21.971437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1031/2100] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:21.980794 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7f561-afef-46b1-8f8c-fefa7787e637 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:21.981068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7f561-afef-46b1-8f8c-fefa7787e637 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:21.981334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7f561-afef-46b1-8f8c-fefa7787e637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.001584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-41d7f561-afef-46b1-8f8c-fefa7787e637 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:22.002769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-41d7f561-afef-46b1-8f8c-fefa7787e637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:22.010337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1070/2101] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:22.014497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d38cda-e568-483a-a860-bd380c55eec4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.024030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d38cda-e568-483a-a860-bd380c55eec4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.028557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a52f8-6708-44ee-9dcb-b0202edd68e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.029137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a52f8-6708-44ee-9dcb-b0202edd68e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.029410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a52f8-6708-44ee-9dcb-b0202edd68e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.029675 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a52f8-6708-44ee-9dcb-b0202edd68e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.030306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a52f8-6708-44ee-9dcb-b0202edd68e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:31:22.030599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a52f8-6708-44ee-9dcb-b0202edd68e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.032217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a52f8-6708-44ee-9dcb-b0202edd68e6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.044166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1032/2102] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:31:22.048834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7f561-afef-46b1-8f8c-fefa7787e637 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.057881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7f561-afef-46b1-8f8c-fefa7787e637 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.062494 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dd65c-9053-4715-9965-ebb7eaa366f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.063082 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dd65c-9053-4715-9965-ebb7eaa366f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.063993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dd65c-9053-4715-9965-ebb7eaa366f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.064266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dd65c-9053-4715-9965-ebb7eaa366f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.064929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dd65c-9053-4715-9965-ebb7eaa366f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.066797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dd65c-9053-4715-9965-ebb7eaa366f8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.087228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:22.097376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1071/2103] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:22 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:22.098781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a52f8-6708-44ee-9dcb-b0202edd68e6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.108873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a52f8-6708-44ee-9dcb-b0202edd68e6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.164290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8f7bc-1c59-4ff0-bf8f-03e5bf94cdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.164913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8f7bc-1c59-4ff0-bf8f-03e5bf94cdbe admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.165189 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8f7bc-1c59-4ff0-bf8f-03e5bf94cdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.165456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8f7bc-1c59-4ff0-bf8f-03e5bf94cdbe admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.166081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8f7bc-1c59-4ff0-bf8f-03e5bf94cdbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.167970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8f7bc-1c59-4ff0-bf8f-03e5bf94cdbe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.189549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1033/2104] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:22 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:22.193591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dd65c-9053-4715-9965-ebb7eaa366f8 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.203931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dd65c-9053-4715-9965-ebb7eaa366f8 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.208536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720d0f0-56b1-4206-bdc0-40ba72ed51df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.209133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d0f0-56b1-4206-bdc0-40ba72ed51df admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.209415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d0f0-56b1-4206-bdc0-40ba72ed51df admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.209685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d0f0-56b1-4206-bdc0-40ba72ed51df admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.210328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720d0f0-56b1-4206-bdc0-40ba72ed51df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.211915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720d0f0-56b1-4206-bdc0-40ba72ed51df admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.221848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1072/2105] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:22.225756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8f7bc-1c59-4ff0-bf8f-03e5bf94cdbe admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.235830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8f7bc-1c59-4ff0-bf8f-03e5bf94cdbe admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.240519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92e8ce-c961-4d8a-899e-37ef9f4b6999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.241092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92e8ce-c961-4d8a-899e-37ef9f4b6999 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.241371 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92e8ce-c961-4d8a-899e-37ef9f4b6999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.241640 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92e8ce-c961-4d8a-899e-37ef9f4b6999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.242296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92e8ce-c961-4d8a-899e-37ef9f4b6999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.243856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92e8ce-c961-4d8a-899e-37ef9f4b6999 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.254684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1034/2106] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:22.258523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720d0f0-56b1-4206-bdc0-40ba72ed51df admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.268011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720d0f0-56b1-4206-bdc0-40ba72ed51df admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.272617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a448f-28d7-490c-924e-047fa96a723a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.273269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a448f-28d7-490c-924e-047fa96a723a admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.273549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a448f-28d7-490c-924e-047fa96a723a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.273817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a448f-28d7-490c-924e-047fa96a723a admin admin] PATH_INFO: `/v3/projects/502463ddb8c0428aaf1b17b5685dcab8/users/377a3699c5bf4a3b9820b681e557d15b/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.288116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a448f-28d7-490c-924e-047fa96a723a admin admin] RBAC: Authorizing `identity:create_grant(project_id=502463ddb8c0428aaf1b17b5685dcab8, user_id=377a3699c5bf4a3b9820b681e557d15b, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.290788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a448f-28d7-490c-924e-047fa96a723a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.303948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1073/2107] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:22 2022] PUT /identity/v3/projects/502463ddb8c0428aaf1b17b5685dcab8/users/377a3699c5bf4a3b9820b681e557d15b/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:22.315236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da11bb-0275-43cf-84d2-9baebc620da3 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.315541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da11bb-0275-43cf-84d2-9baebc620da3 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.315940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da11bb-0275-43cf-84d2-9baebc620da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.342654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-35da11bb-0275-43cf-84d2-9baebc620da3 None None] MFA Rules not processed for user `377a3699c5bf4a3b9820b681e557d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:22.343859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-35da11bb-0275-43cf-84d2-9baebc620da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:22.412377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1035/2108] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:22.416847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a448f-28d7-490c-924e-047fa96a723a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.478017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a448f-28d7-490c-924e-047fa96a723a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.482666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb0c0e-3f70-4153-a04c-e7b9bc94030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.483265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb0c0e-3f70-4153-a04c-e7b9bc94030b admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.483539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb0c0e-3f70-4153-a04c-e7b9bc94030b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.483809 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb0c0e-3f70-4153-a04c-e7b9bc94030b admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.484470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb0c0e-3f70-4153-a04c-e7b9bc94030b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.486978 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb0c0e-3f70-4153-a04c-e7b9bc94030b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.517204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1074/2109] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:22 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:22.520897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da11bb-0275-43cf-84d2-9baebc620da3 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.531511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da11bb-0275-43cf-84d2-9baebc620da3 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.560971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:22.597043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f118c8-7e11-42cc-9d05-c921307a19b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.597781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f118c8-7e11-42cc-9d05-c921307a19b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.598129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f118c8-7e11-42cc-9d05-c921307a19b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.598408 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f118c8-7e11-42cc-9d05-c921307a19b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.599210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f118c8-7e11-42cc-9d05-c921307a19b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.601271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f118c8-7e11-42cc-9d05-c921307a19b8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.628912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1036/2110] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:22 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:22.629877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb0c0e-3f70-4153-a04c-e7b9bc94030b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.642195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb0c0e-3f70-4153-a04c-e7b9bc94030b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.647339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d35b9-1c10-4953-8b15-dd54734f468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.647966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d35b9-1c10-4953-8b15-dd54734f468f admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.648241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d35b9-1c10-4953-8b15-dd54734f468f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.648596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d35b9-1c10-4953-8b15-dd54734f468f admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.649236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d35b9-1c10-4953-8b15-dd54734f468f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.650834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d35b9-1c10-4953-8b15-dd54734f468f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.665748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1075/2111] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:22.670102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f118c8-7e11-42cc-9d05-c921307a19b8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.681243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f118c8-7e11-42cc-9d05-c921307a19b8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.685978 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7083c-e619-499b-afec-89157247ccb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.686738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7083c-e619-499b-afec-89157247ccb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.687096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7083c-e619-499b-afec-89157247ccb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.687432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7083c-e619-499b-afec-89157247ccb2 admin admin] PATH_INFO: `/v3/projects/6575d4bdc6774a44b5816ac8e6fc80af/users/f25d51d26c5a4c7a822badec744c439b/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.703065 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7083c-e619-499b-afec-89157247ccb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6575d4bdc6774a44b5816ac8e6fc80af, user_id=f25d51d26c5a4c7a822badec744c439b, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.706300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7083c-e619-499b-afec-89157247ccb2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.721319 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1037/2112] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:22 2022] PUT /identity/v3/projects/6575d4bdc6774a44b5816ac8e6fc80af/users/f25d51d26c5a4c7a822badec744c439b/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:22.729455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d35b9-1c10-4953-8b15-dd54734f468f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.741973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d35b9-1c10-4953-8b15-dd54734f468f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.799337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e69b0-619b-4696-9c16-6ae899a910ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.799988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e69b0-619b-4696-9c16-6ae899a910ea admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.800197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e69b0-619b-4696-9c16-6ae899a910ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.800496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e69b0-619b-4696-9c16-6ae899a910ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.801130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e69b0-619b-4696-9c16-6ae899a910ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.802692 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e69b0-619b-4696-9c16-6ae899a910ea admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.812458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1076/2113] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:22.816661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7083c-e619-499b-afec-89157247ccb2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.831347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7083c-e619-499b-afec-89157247ccb2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:22.836853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e757e8-55ae-4d1d-bfd8-db379ea67c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:22.837553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e757e8-55ae-4d1d-bfd8-db379ea67c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.837832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e757e8-55ae-4d1d-bfd8-db379ea67c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.838107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e757e8-55ae-4d1d-bfd8-db379ea67c9c admin admin] PATH_INFO: `/v3/projects/6575d4bdc6774a44b5816ac8e6fc80af/users/f25d51d26c5a4c7a822badec744c439b/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.840016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e757e8-55ae-4d1d-bfd8-db379ea67c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6575d4bdc6774a44b5816ac8e6fc80af, user_id=f25d51d26c5a4c7a822badec744c439b, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:22.842688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e757e8-55ae-4d1d-bfd8-db379ea67c9c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:22.853537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1038/2114] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:22 2022] PUT /identity/v3/projects/6575d4bdc6774a44b5816ac8e6fc80af/users/f25d51d26c5a4c7a822badec744c439b/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:22.863754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8d73c-aced-47c8-acf1-092951fc902a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:22.864046 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8d73c-aced-47c8-acf1-092951fc902a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:22.864313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8d73c-aced-47c8-acf1-092951fc902a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:22.891051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-19f8d73c-aced-47c8-acf1-092951fc902a None None] MFA Rules not processed for user `f25d51d26c5a4c7a822badec744c439b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:22.892765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-19f8d73c-aced-47c8-acf1-092951fc902a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:22.916781 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:22.960722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1077/2115] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:22.971778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e757e8-55ae-4d1d-bfd8-db379ea67c9c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:22.973616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-82e757e8-55ae-4d1d-bfd8-db379ea67c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:23.029410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e757e8-55ae-4d1d-bfd8-db379ea67c9c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:23.033628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c63482-bb9c-43cf-8b2a-86629dc407c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:23.034484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c63482-bb9c-43cf-8b2a-86629dc407c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:23.034761 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c63482-bb9c-43cf-8b2a-86629dc407c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:23.035034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c63482-bb9c-43cf-8b2a-86629dc407c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:23.043867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c63482-bb9c-43cf-8b2a-86629dc407c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:23.045320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c63482-bb9c-43cf-8b2a-86629dc407c3 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:23.054791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1039/2116] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:23.185466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8d73c-aced-47c8-acf1-092951fc902a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:23.245621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8d73c-aced-47c8-acf1-092951fc902a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:23.250020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:23.250633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:23.251416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:23.251681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] PATH_INFO: `/v3/users/377a3699c5bf4a3b9820b681e557d15b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:23.252806 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=377a3699c5bf4a3b9820b681e557d15b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:23.254698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:23.300938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377a3699c5bf4a3b9820b681e557d15b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:23.307826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377a3699c5bf4a3b9820b681e557d15b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:23.308524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377a3699c5bf4a3b9820b681e557d15b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:23.316282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377a3699c5bf4a3b9820b681e557d15b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:23.328557 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377a3699c5bf4a3b9820b681e557d15b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:23.335019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1078/2117] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:23 2022] DELETE /identity/v3/users/377a3699c5bf4a3b9820b681e557d15b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:23.340290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786d771-9796-4f53-98b3-341a50f86229 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:23.340595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786d771-9796-4f53-98b3-341a50f86229 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:23.340865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786d771-9796-4f53-98b3-341a50f86229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:23.358957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9786d771-9796-4f53-98b3-341a50f86229 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:23.360472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9786d771-9796-4f53-98b3-341a50f86229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:23.421377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1040/2118] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:31:23.431543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:23.432971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:23.477923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:23.492845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d1046-a2f8-4505-9e2d-4574d85b3b0a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:23.496495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861b0ec-0ab8-42cf-8d87-563771b80c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:23.497095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861b0ec-0ab8-42cf-8d87-563771b80c2f service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:23.497372 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861b0ec-0ab8-42cf-8d87-563771b80c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:23.497642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861b0ec-0ab8-42cf-8d87-563771b80c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:23.507028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861b0ec-0ab8-42cf-8d87-563771b80c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:23.508480 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861b0ec-0ab8-42cf-8d87-563771b80c2f service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:23.517587 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1079/2119] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:23.791902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786d771-9796-4f53-98b3-341a50f86229 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:23.793394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9786d771-9796-4f53-98b3-341a50f86229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:23.809329 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786d771-9796-4f53-98b3-341a50f86229 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:23.816865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:23.817568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:23.817876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:23.818126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] PATH_INFO: `/v3/projects/502463ddb8c0428aaf1b17b5685dcab8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:23.819898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=502463ddb8c0428aaf1b17b5685dcab8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:23.821719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:23.848654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:23.895998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '502463ddb8c0428aaf1b17b5685dcab8', 'request_id': 'req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:23.903053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '502463ddb8c0428aaf1b17b5685dcab8', 'request_id': 'req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:23.910216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '502463ddb8c0428aaf1b17b5685dcab8', 'request_id': 'req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:23.923928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] The token cache is being invalidate because project 502463ddb8c0428aaf1b17b5685dcab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:23.924246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:23.924946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:23.926536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1041/2120] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:23 2022] DELETE /identity/v3/projects/502463ddb8c0428aaf1b17b5685dcab8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:23.930572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861b0ec-0ab8-42cf-8d87-563771b80c2f service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:23.940890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861b0ec-0ab8-42cf-8d87-563771b80c2f service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:23.997404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de6ec4-d1ef-4d54-8bb9-c5af85585288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:23.998033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6ec4-d1ef-4d54-8bb9-c5af85585288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:23.998308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6ec4-d1ef-4d54-8bb9-c5af85585288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:23.998577 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6ec4-d1ef-4d54-8bb9-c5af85585288 admin admin] PATH_INFO: `/v3/users/f25d51d26c5a4c7a822badec744c439b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:23.999718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de6ec4-d1ef-4d54-8bb9-c5af85585288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25d51d26c5a4c7a822badec744c439b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:24.001814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de6ec4-d1ef-4d54-8bb9-c5af85585288 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:24.049583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-69de6ec4-d1ef-4d54-8bb9-c5af85585288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25d51d26c5a4c7a822badec744c439b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:24.057081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-69de6ec4-d1ef-4d54-8bb9-c5af85585288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25d51d26c5a4c7a822badec744c439b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:24.058085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-69de6ec4-d1ef-4d54-8bb9-c5af85585288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25d51d26c5a4c7a822badec744c439b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:24.066620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-69de6ec4-d1ef-4d54-8bb9-c5af85585288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25d51d26c5a4c7a822badec744c439b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:24.079561 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-69de6ec4-d1ef-4d54-8bb9-c5af85585288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25d51d26c5a4c7a822badec744c439b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:24.085437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1080/2121] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:23 2022] DELETE /identity/v3/users/f25d51d26c5a4c7a822badec744c439b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:31:24.440992 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:24.442458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:24.455873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ff5ef-66fc-4a19-a11e-969b210ffa8f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:24.513575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:24.514204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:24.514477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:24.514742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] PATH_INFO: `/v3/projects/6575d4bdc6774a44b5816ac8e6fc80af` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:24.515824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6575d4bdc6774a44b5816ac8e6fc80af)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:24.518357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:24.564330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6575d4bdc6774a44b5816ac8e6fc80af', 'request_id': 'req-befbd0c8-bf6d-4835-a93b-0635933fd085'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:24.569791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6575d4bdc6774a44b5816ac8e6fc80af', 'request_id': 'req-befbd0c8-bf6d-4835-a93b-0635933fd085'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:24.575689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6575d4bdc6774a44b5816ac8e6fc80af', 'request_id': 'req-befbd0c8-bf6d-4835-a93b-0635933fd085'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:24.588525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] The token cache is being invalidate because project 6575d4bdc6774a44b5816ac8e6fc80af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:24.588864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:24.589485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-befbd0c8-bf6d-4835-a93b-0635933fd085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:24.590996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1042/2122] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:24 2022] DELETE /identity/v3/projects/6575d4bdc6774a44b5816ac8e6fc80af => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:24.595989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21138a5e-679b-43fb-895c-f56ebe15a86d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:24.596278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21138a5e-679b-43fb-895c-f56ebe15a86d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:24.596582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21138a5e-679b-43fb-895c-f56ebe15a86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:24.615725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-21138a5e-679b-43fb-895c-f56ebe15a86d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:24.616959 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-21138a5e-679b-43fb-895c-f56ebe15a86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:24.674537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1081/2123] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:24.683255 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c514c-932a-4fc4-97fc-b551ee06b8ee None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:24.683590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c514c-932a-4fc4-97fc-b551ee06b8ee None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:24.683813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c514c-932a-4fc4-97fc-b551ee06b8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:24.703333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7a3c514c-932a-4fc4-97fc-b551ee06b8ee None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:24.705183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7a3c514c-932a-4fc4-97fc-b551ee06b8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:24.712897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1043/2124] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:24.716875 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21138a5e-679b-43fb-895c-f56ebe15a86d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:24.721594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:24.728131 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21138a5e-679b-43fb-895c-f56ebe15a86d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:24.733850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac836307-81c7-4ec5-9d13-c4161e36f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:24.734458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac836307-81c7-4ec5-9d13-c4161e36f17e admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:24.734739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac836307-81c7-4ec5-9d13-c4161e36f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:24.735015 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac836307-81c7-4ec5-9d13-c4161e36f17e admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:24.735664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac836307-81c7-4ec5-9d13-c4161e36f17e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:31:24.735969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac836307-81c7-4ec5-9d13-c4161e36f17e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:24.737635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac836307-81c7-4ec5-9d13-c4161e36f17e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:24.749389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1082/2125] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:31:24.754232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c514c-932a-4fc4-97fc-b551ee06b8ee None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:24.763388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c514c-932a-4fc4-97fc-b551ee06b8ee None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:24.768674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844ff1b-27c0-454f-93db-8ca6dac4299a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:24.769281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844ff1b-27c0-454f-93db-8ca6dac4299a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:24.769547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844ff1b-27c0-454f-93db-8ca6dac4299a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:24.769816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844ff1b-27c0-454f-93db-8ca6dac4299a admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:24.770479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844ff1b-27c0-454f-93db-8ca6dac4299a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:24.772358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844ff1b-27c0-454f-93db-8ca6dac4299a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:24.800547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1044/2126] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:24 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:24.804277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac836307-81c7-4ec5-9d13-c4161e36f17e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:24.815036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac836307-81c7-4ec5-9d13-c4161e36f17e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:24.873927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8b5e2-9e2d-4a5b-89af-7fe3015a9d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:24.874640 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8b5e2-9e2d-4a5b-89af-7fe3015a9d40 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:24.875028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8b5e2-9e2d-4a5b-89af-7fe3015a9d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:24.875437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8b5e2-9e2d-4a5b-89af-7fe3015a9d40 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:24.876768 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8b5e2-9e2d-4a5b-89af-7fe3015a9d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:24.879413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8b5e2-9e2d-4a5b-89af-7fe3015a9d40 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:24.904287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1083/2127] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:24 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:24.908348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844ff1b-27c0-454f-93db-8ca6dac4299a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:24.919851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844ff1b-27c0-454f-93db-8ca6dac4299a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:24.926373 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd101d7-f7de-4049-b719-d2c9758822e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:24.927242 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd101d7-f7de-4049-b719-d2c9758822e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:24.927667 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd101d7-f7de-4049-b719-d2c9758822e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:24.928025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd101d7-f7de-4049-b719-d2c9758822e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:24.928868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd101d7-f7de-4049-b719-d2c9758822e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:24.931136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd101d7-f7de-4049-b719-d2c9758822e4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:24.934154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:24.946554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1045/2128] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:24.950464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8b5e2-9e2d-4a5b-89af-7fe3015a9d40 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:24.972080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8b5e2-9e2d-4a5b-89af-7fe3015a9d40 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:24.982351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcecb2f-f46c-4594-ace7-ed4cbb92e7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:24.983168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcecb2f-f46c-4594-ace7-ed4cbb92e7de admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:24.983594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcecb2f-f46c-4594-ace7-ed4cbb92e7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:24.983958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcecb2f-f46c-4594-ace7-ed4cbb92e7de admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:24.984756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcecb2f-f46c-4594-ace7-ed4cbb92e7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:24.987330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcecb2f-f46c-4594-ace7-ed4cbb92e7de admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:25.000078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1084/2129] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:25.005351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd101d7-f7de-4049-b719-d2c9758822e4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:25.016305 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd101d7-f7de-4049-b719-d2c9758822e4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:25.029128 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c336bf-6b47-49d4-8889-d730d443291d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:25.029871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c336bf-6b47-49d4-8889-d730d443291d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:25.030215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c336bf-6b47-49d4-8889-d730d443291d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:25.030551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c336bf-6b47-49d4-8889-d730d443291d admin admin] PATH_INFO: `/v3/projects/543283c48e084fb2b865ee8aab34a81d/users/e43a19229b4e462190defc46d35721b1/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:25.046692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c336bf-6b47-49d4-8889-d730d443291d admin admin] RBAC: Authorizing `identity:create_grant(project_id=543283c48e084fb2b865ee8aab34a81d, user_id=e43a19229b4e462190defc46d35721b1, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:25.050027 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c336bf-6b47-49d4-8889-d730d443291d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:25.064494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1046/2130] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:25 2022] PUT /identity/v3/projects/543283c48e084fb2b865ee8aab34a81d/users/e43a19229b4e462190defc46d35721b1/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:25.074608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833256d8-3d85-40fe-affa-e5a7baaacd28 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:25.074897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833256d8-3d85-40fe-affa-e5a7baaacd28 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:25.075174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833256d8-3d85-40fe-affa-e5a7baaacd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:25.103278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-833256d8-3d85-40fe-affa-e5a7baaacd28 None None] MFA Rules not processed for user `e43a19229b4e462190defc46d35721b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:25.104542 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-833256d8-3d85-40fe-affa-e5a7baaacd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:25.175207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1085/2131] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:25.179729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c336bf-6b47-49d4-8889-d730d443291d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:25.238827 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c336bf-6b47-49d4-8889-d730d443291d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:25.243262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaead2-1e50-4f8d-af60-230cc84c9f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:25.243856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaead2-1e50-4f8d-af60-230cc84c9f4c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:25.244132 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaead2-1e50-4f8d-af60-230cc84c9f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:25.244427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaead2-1e50-4f8d-af60-230cc84c9f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:25.245048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaead2-1e50-4f8d-af60-230cc84c9f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:25.246962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaead2-1e50-4f8d-af60-230cc84c9f4c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:25.272715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1047/2132] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:25 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:25.276715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833256d8-3d85-40fe-affa-e5a7baaacd28 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:25.287367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833256d8-3d85-40fe-affa-e5a7baaacd28 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:25.338302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:25.345902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c0933-f3f1-46ea-b40b-18e4d827d189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:25.346735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c0933-f3f1-46ea-b40b-18e4d827d189 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:25.347196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c0933-f3f1-46ea-b40b-18e4d827d189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:25.347650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c0933-f3f1-46ea-b40b-18e4d827d189 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:25.348425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c0933-f3f1-46ea-b40b-18e4d827d189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:25.350396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c0933-f3f1-46ea-b40b-18e4d827d189 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:25.374800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1086/2133] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:25 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:25.376404 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaead2-1e50-4f8d-af60-230cc84c9f4c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:25.389088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaead2-1e50-4f8d-af60-230cc84c9f4c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:25.393768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a8f84-07d4-42ef-8351-3b0fa325bd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:25.394365 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a8f84-07d4-42ef-8351-3b0fa325bd5a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:25.394646 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a8f84-07d4-42ef-8351-3b0fa325bd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:25.394946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a8f84-07d4-42ef-8351-3b0fa325bd5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:25.395615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a8f84-07d4-42ef-8351-3b0fa325bd5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:25.399352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a8f84-07d4-42ef-8351-3b0fa325bd5a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:25.409978 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1048/2134] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:31:25.414227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c0933-f3f1-46ea-b40b-18e4d827d189 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:25.424570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c0933-f3f1-46ea-b40b-18e4d827d189 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:25.429584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f64371-e424-49d2-a3f2-5436dbe7b7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:25.430232 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f64371-e424-49d2-a3f2-5436dbe7b7a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:25.430506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f64371-e424-49d2-a3f2-5436dbe7b7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:25.430771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f64371-e424-49d2-a3f2-5436dbe7b7a4 admin admin] PATH_INFO: `/v3/projects/182e189503194700aeb58611c6cd0f96/users/e14e5822e66d4f20b3e440927ccd9de9/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:25.446978 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f64371-e424-49d2-a3f2-5436dbe7b7a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=182e189503194700aeb58611c6cd0f96, user_id=e14e5822e66d4f20b3e440927ccd9de9, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:25.449881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f64371-e424-49d2-a3f2-5436dbe7b7a4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:25.471061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1087/2135] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:25 2022] PUT /identity/v3/projects/182e189503194700aeb58611c6cd0f96/users/e14e5822e66d4f20b3e440927ccd9de9/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:25.475026 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a8f84-07d4-42ef-8351-3b0fa325bd5a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:25.485032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a8f84-07d4-42ef-8351-3b0fa325bd5a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:25.540504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bebd0-bf88-476a-8cbb-3c7ca65406e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:25.541034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bebd0-bf88-476a-8cbb-3c7ca65406e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:25.541314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bebd0-bf88-476a-8cbb-3c7ca65406e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:25.541583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bebd0-bf88-476a-8cbb-3c7ca65406e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:25.542215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bebd0-bf88-476a-8cbb-3c7ca65406e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:25.543799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bebd0-bf88-476a-8cbb-3c7ca65406e1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:25.553851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1049/2136] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:25.558049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f64371-e424-49d2-a3f2-5436dbe7b7a4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:25.568122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f64371-e424-49d2-a3f2-5436dbe7b7a4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:25.572632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03749f26-c036-4fdc-9de8-b3650f7a9975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:25.573264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03749f26-c036-4fdc-9de8-b3650f7a9975 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:25.573535 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03749f26-c036-4fdc-9de8-b3650f7a9975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:25.573798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03749f26-c036-4fdc-9de8-b3650f7a9975 admin admin] PATH_INFO: `/v3/projects/182e189503194700aeb58611c6cd0f96/users/e14e5822e66d4f20b3e440927ccd9de9/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:25.575651 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03749f26-c036-4fdc-9de8-b3650f7a9975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=182e189503194700aeb58611c6cd0f96, user_id=e14e5822e66d4f20b3e440927ccd9de9, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:25.578244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03749f26-c036-4fdc-9de8-b3650f7a9975 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:25.587909 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1088/2137] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:25 2022] PUT /identity/v3/projects/182e189503194700aeb58611c6cd0f96/users/e14e5822e66d4f20b3e440927ccd9de9/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:31:25.599054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3d3d0-bda4-4e63-88cd-58ddb9d4a99c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:25.599341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3d3d0-bda4-4e63-88cd-58ddb9d4a99c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:25.599611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3d3d0-bda4-4e63-88cd-58ddb9d4a99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:25.627203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a9c3d3d0-bda4-4e63-88cd-58ddb9d4a99c None None] MFA Rules not processed for user `e14e5822e66d4f20b3e440927ccd9de9`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:25.628416 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a9c3d3d0-bda4-4e63-88cd-58ddb9d4a99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:25.659809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:25.696677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1050/2138] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:25.708929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03749f26-c036-4fdc-9de8-b3650f7a9975 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:25.710612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-03749f26-c036-4fdc-9de8-b3650f7a9975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:25.767565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03749f26-c036-4fdc-9de8-b3650f7a9975 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:25.771211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f1935-0eee-48e1-8e61-100a316be330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:25.771806 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f1935-0eee-48e1-8e61-100a316be330 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:25.772082 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f1935-0eee-48e1-8e61-100a316be330 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:25.772350 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f1935-0eee-48e1-8e61-100a316be330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:25.781174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f1935-0eee-48e1-8e61-100a316be330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:25.783247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f1935-0eee-48e1-8e61-100a316be330 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:25.795893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1089/2139] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:25.886105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3d3d0-bda4-4e63-88cd-58ddb9d4a99c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:25.896181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3d3d0-bda4-4e63-88cd-58ddb9d4a99c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:25.899718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482e103-b319-4af1-b4dd-cf26c30ddc63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:25.900304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482e103-b319-4af1-b4dd-cf26c30ddc63 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:25.900614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482e103-b319-4af1-b4dd-cf26c30ddc63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:25.900885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482e103-b319-4af1-b4dd-cf26c30ddc63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:25.957634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482e103-b319-4af1-b4dd-cf26c30ddc63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:25.959122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482e103-b319-4af1-b4dd-cf26c30ddc63 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:25.967861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1051/2140] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:26.257820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f1935-0eee-48e1-8e61-100a316be330 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:26.319863 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f1935-0eee-48e1-8e61-100a316be330 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:26.324490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:26.325139 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:26.325439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:26.325710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] PATH_INFO: `/v3/users/e43a19229b4e462190defc46d35721b1` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:26.326804 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43a19229b4e462190defc46d35721b1)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:26.328764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:26.374005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43a19229b4e462190defc46d35721b1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:26.379705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:26.381801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43a19229b4e462190defc46d35721b1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:26.382721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43a19229b4e462190defc46d35721b1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:26.391151 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43a19229b4e462190defc46d35721b1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:26.404115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43a19229b4e462190defc46d35721b1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:26.410200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1090/2141] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:26 2022] DELETE /identity/v3/users/e43a19229b4e462190defc46d35721b1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:26.414599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f968231-c4d8-4913-9dc6-1a43955c1460 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:26.414881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f968231-c4d8-4913-9dc6-1a43955c1460 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:26.415144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f968231-c4d8-4913-9dc6-1a43955c1460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:26.433934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4f968231-c4d8-4913-9dc6-1a43955c1460 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:26.435112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4f968231-c4d8-4913-9dc6-1a43955c1460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:26.492766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1052/2142] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:26.502839 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:26.504248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:26.562252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b27f1-c7a4-4ea3-b0ae-5aeecfda8eb7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:26.565951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0029e-525b-4d59-aaee-b5dd256454cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:26.566561 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0029e-525b-4d59-aaee-b5dd256454cf service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:26.566869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0029e-525b-4d59-aaee-b5dd256454cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:26.567087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0029e-525b-4d59-aaee-b5dd256454cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:26.576274 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0029e-525b-4d59-aaee-b5dd256454cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:26.577758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0029e-525b-4d59-aaee-b5dd256454cf service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:26.587891 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1091/2143] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:26.857410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f968231-c4d8-4913-9dc6-1a43955c1460 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:26.858928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4f968231-c4d8-4913-9dc6-1a43955c1460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:26.872648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f968231-c4d8-4913-9dc6-1a43955c1460 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:26.877546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:26.878865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:26.879502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:26.879782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] PATH_INFO: `/v3/projects/543283c48e084fb2b865ee8aab34a81d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:26.880983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=543283c48e084fb2b865ee8aab34a81d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:26.882805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:26.929307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '543283c48e084fb2b865ee8aab34a81d', 'request_id': 'req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:26.934769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '543283c48e084fb2b865ee8aab34a81d', 'request_id': 'req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:26.940730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '543283c48e084fb2b865ee8aab34a81d', 'request_id': 'req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:26.942349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:26.953612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] The token cache is being invalidate because project 543283c48e084fb2b865ee8aab34a81d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:26.953947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:26.954584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:26.956232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1053/2144] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:26 2022] DELETE /identity/v3/projects/543283c48e084fb2b865ee8aab34a81d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:26.960120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0029e-525b-4d59-aaee-b5dd256454cf service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:26.970287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0029e-525b-4d59-aaee-b5dd256454cf service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:27.025175 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc46f9-b736-42c8-9077-375a735823d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:27.025795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc46f9-b736-42c8-9077-375a735823d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:27.026069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc46f9-b736-42c8-9077-375a735823d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:27.026336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc46f9-b736-42c8-9077-375a735823d5 admin admin] PATH_INFO: `/v3/users/e14e5822e66d4f20b3e440927ccd9de9` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:27.027448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc46f9-b736-42c8-9077-375a735823d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e14e5822e66d4f20b3e440927ccd9de9)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:27.029383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc46f9-b736-42c8-9077-375a735823d5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:27.077297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a8dc46f9-b736-42c8-9077-375a735823d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e14e5822e66d4f20b3e440927ccd9de9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:27.084025 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a8dc46f9-b736-42c8-9077-375a735823d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e14e5822e66d4f20b3e440927ccd9de9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:27.084706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a8dc46f9-b736-42c8-9077-375a735823d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e14e5822e66d4f20b3e440927ccd9de9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:27.092093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a8dc46f9-b736-42c8-9077-375a735823d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e14e5822e66d4f20b3e440927ccd9de9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:27.103157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:27.105207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a8dc46f9-b736-42c8-9077-375a735823d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e14e5822e66d4f20b3e440927ccd9de9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:27.113757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1092/2145] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:26 2022] DELETE /identity/v3/users/e14e5822e66d4f20b3e440927ccd9de9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:27.436956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:27.438427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:27.451830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2cce7-d0ba-4a92-9fcb-6395f06e14a0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:27.516799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:27.517476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:27.517719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:27.517986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] PATH_INFO: `/v3/projects/182e189503194700aeb58611c6cd0f96` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:27.519108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=182e189503194700aeb58611c6cd0f96)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:27.521022 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:27.566293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182e189503194700aeb58611c6cd0f96', 'request_id': 'req-2b318be4-bd77-4005-95a2-88dd002cd997'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:27.571605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182e189503194700aeb58611c6cd0f96', 'request_id': 'req-2b318be4-bd77-4005-95a2-88dd002cd997'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:27.577497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182e189503194700aeb58611c6cd0f96', 'request_id': 'req-2b318be4-bd77-4005-95a2-88dd002cd997'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:27.589849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] The token cache is being invalidate because project 182e189503194700aeb58611c6cd0f96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:27.590134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:27.590743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2b318be4-bd77-4005-95a2-88dd002cd997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:27.592230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1054/2146] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:27 2022] DELETE /identity/v3/projects/182e189503194700aeb58611c6cd0f96 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:27.597148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07cb8a-a4db-41e9-922b-20aa2acdf416 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:27.597627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07cb8a-a4db-41e9-922b-20aa2acdf416 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:27.597827 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07cb8a-a4db-41e9-922b-20aa2acdf416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:27.617180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7e07cb8a-a4db-41e9-922b-20aa2acdf416 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:27.618370 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7e07cb8a-a4db-41e9-922b-20aa2acdf416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:27.676416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1093/2147] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:27.684184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022246f8-4a7c-4264-9775-08d31b2b762a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:27.684514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022246f8-4a7c-4264-9775-08d31b2b762a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:27.684793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022246f8-4a7c-4264-9775-08d31b2b762a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:27.705426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-022246f8-4a7c-4264-9775-08d31b2b762a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:27.706662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-022246f8-4a7c-4264-9775-08d31b2b762a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:27.714933 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1055/2148] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:27.718996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07cb8a-a4db-41e9-922b-20aa2acdf416 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:27.728864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07cb8a-a4db-41e9-922b-20aa2acdf416 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:27.733468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4314f0-1739-4fd9-979c-c3b66c3420df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:27.734067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4314f0-1739-4fd9-979c-c3b66c3420df admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:27.734345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4314f0-1739-4fd9-979c-c3b66c3420df admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:27.734614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4314f0-1739-4fd9-979c-c3b66c3420df admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:27.735262 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4314f0-1739-4fd9-979c-c3b66c3420df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:31:27.735566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4314f0-1739-4fd9-979c-c3b66c3420df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:27.737198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4314f0-1739-4fd9-979c-c3b66c3420df admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:27.748827 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1094/2149] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:31:27.753715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022246f8-4a7c-4264-9775-08d31b2b762a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:27.763107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022246f8-4a7c-4264-9775-08d31b2b762a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:27.767906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af702785-1286-46b7-895e-ec730b54aa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:27.768534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af702785-1286-46b7-895e-ec730b54aa9f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:27.768812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af702785-1286-46b7-895e-ec730b54aa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:27.769081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af702785-1286-46b7-895e-ec730b54aa9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:27.769713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af702785-1286-46b7-895e-ec730b54aa9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:27.771601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af702785-1286-46b7-895e-ec730b54aa9f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:27.800863 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1056/2150] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:27 2022] POST /identity/v3/projects => generated 359 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:27.805507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4314f0-1739-4fd9-979c-c3b66c3420df admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:27.815597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4314f0-1739-4fd9-979c-c3b66c3420df admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:27.877905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33185a43-b952-440b-87f5-8f4a2b58e6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:27.879034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33185a43-b952-440b-87f5-8f4a2b58e6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:27.879345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33185a43-b952-440b-87f5-8f4a2b58e6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:27.879624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33185a43-b952-440b-87f5-8f4a2b58e6f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:27.880323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33185a43-b952-440b-87f5-8f4a2b58e6f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:27.882559 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33185a43-b952-440b-87f5-8f4a2b58e6f7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:27.904386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1095/2151] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:27 2022] POST /identity/v3/users => generated 280 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:27.908552 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af702785-1286-46b7-895e-ec730b54aa9f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:27.919087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af702785-1286-46b7-895e-ec730b54aa9f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:27.924127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f74eb-b4a4-49f8-b0ac-35b66b83b70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:27.924767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f74eb-b4a4-49f8-b0ac-35b66b83b70b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:27.925045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f74eb-b4a4-49f8-b0ac-35b66b83b70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:27.925316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f74eb-b4a4-49f8-b0ac-35b66b83b70b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:27.925962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f74eb-b4a4-49f8-b0ac-35b66b83b70b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:27.927543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f74eb-b4a4-49f8-b0ac-35b66b83b70b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:27.937344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1057/2152] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:27.942089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33185a43-b952-440b-87f5-8f4a2b58e6f7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:27.951833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33185a43-b952-440b-87f5-8f4a2b58e6f7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:27.956527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9272e-bcb5-43a9-85ed-f9bd3979087d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:27.957145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9272e-bcb5-43a9-85ed-f9bd3979087d admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:27.957423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9272e-bcb5-43a9-85ed-f9bd3979087d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:27.957697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9272e-bcb5-43a9-85ed-f9bd3979087d admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:27.958336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9272e-bcb5-43a9-85ed-f9bd3979087d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:27.959910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9272e-bcb5-43a9-85ed-f9bd3979087d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:27.970465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1096/2153] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:31:27.974482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f74eb-b4a4-49f8-b0ac-35b66b83b70b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:27.984486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f74eb-b4a4-49f8-b0ac-35b66b83b70b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:27.989512 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53d74-71d4-4d87-a90c-c9d2157c4dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:27.990142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53d74-71d4-4d87-a90c-c9d2157c4dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:27.990421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53d74-71d4-4d87-a90c-c9d2157c4dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:27.990695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53d74-71d4-4d87-a90c-c9d2157c4dbf admin admin] PATH_INFO: `/v3/projects/7e853efaf369401581938b17076841bd/users/767b6dbed29f43e5b77584a2d65edf3f/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:28.005736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d53d74-71d4-4d87-a90c-c9d2157c4dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e853efaf369401581938b17076841bd, user_id=767b6dbed29f43e5b77584a2d65edf3f, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:28.008358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d53d74-71d4-4d87-a90c-c9d2157c4dbf admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:28.020941 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1058/2154] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:27 2022] PUT /identity/v3/projects/7e853efaf369401581938b17076841bd/users/767b6dbed29f43e5b77584a2d65edf3f/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:28.031308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b321f47-db4c-4510-845b-c6b505ff3d0b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:28.031588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b321f47-db4c-4510-845b-c6b505ff3d0b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:28.031917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b321f47-db4c-4510-845b-c6b505ff3d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:28.060197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8b321f47-db4c-4510-845b-c6b505ff3d0b None None] MFA Rules not processed for user `767b6dbed29f43e5b77584a2d65edf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:28.061452 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8b321f47-db4c-4510-845b-c6b505ff3d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:28.106694 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:28.127044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1097/2155] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:28.160988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53d74-71d4-4d87-a90c-c9d2157c4dbf admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:28.162685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-87d53d74-71d4-4d87-a90c-c9d2157c4dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:28.183083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:28.226732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53d74-71d4-4d87-a90c-c9d2157c4dbf admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:28.230495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ef4ad-f3c4-47ae-a3f4-c1fe729527d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:28.231163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ef4ad-f3c4-47ae-a3f4-c1fe729527d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:28.231439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ef4ad-f3c4-47ae-a3f4-c1fe729527d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:28.231707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ef4ad-f3c4-47ae-a3f4-c1fe729527d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:28.239871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ef4ad-f3c4-47ae-a3f4-c1fe729527d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:28.241315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ef4ad-f3c4-47ae-a3f4-c1fe729527d5 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:28.249795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1059/2156] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:28.265569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b321f47-db4c-4510-845b-c6b505ff3d0b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:28.332171 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b321f47-db4c-4510-845b-c6b505ff3d0b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:28.336834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:28.337469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:28.337740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:28.338007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] PATH_INFO: `/v3/users/767b6dbed29f43e5b77584a2d65edf3f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:28.341611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] RBAC: Authorizing `identity:delete_user(user_id=767b6dbed29f43e5b77584a2d65edf3f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:28.343492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:28.394493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '767b6dbed29f43e5b77584a2d65edf3f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:28.402245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '767b6dbed29f43e5b77584a2d65edf3f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:28.403000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '767b6dbed29f43e5b77584a2d65edf3f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:28.410750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '767b6dbed29f43e5b77584a2d65edf3f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:28.422867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '767b6dbed29f43e5b77584a2d65edf3f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:28.428620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1098/2157] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:28 2022] DELETE /identity/v3/users/767b6dbed29f43e5b77584a2d65edf3f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:31:28.433286 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df534f33-a767-4fa3-b1a0-f66a24480deb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:28.433517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df534f33-a767-4fa3-b1a0-f66a24480deb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:28.433809 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df534f33-a767-4fa3-b1a0-f66a24480deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:28.452969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-df534f33-a767-4fa3-b1a0-f66a24480deb None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:28.454786 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-df534f33-a767-4fa3-b1a0-f66a24480deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:28.512769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1060/2158] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:28.522695 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:28.524135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:28.583927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d619a-02b0-4b7d-b982-6680a33dc72a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:28.587818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc911f-e8b6-49da-9e60-f24908463133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:28.588395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc911f-e8b6-49da-9e60-f24908463133 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:28.588691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc911f-e8b6-49da-9e60-f24908463133 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:28.588963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc911f-e8b6-49da-9e60-f24908463133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:28.598722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc911f-e8b6-49da-9e60-f24908463133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:28.600140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc911f-e8b6-49da-9e60-f24908463133 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:28.609670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1099/2159] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:28.877844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df534f33-a767-4fa3-b1a0-f66a24480deb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:28.880160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-df534f33-a767-4fa3-b1a0-f66a24480deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:28.893318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df534f33-a767-4fa3-b1a0-f66a24480deb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:28.898254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:28.898949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:28.899294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:28.899654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] PATH_INFO: `/v3/projects/7e853efaf369401581938b17076841bd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:28.900934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e853efaf369401581938b17076841bd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:28.903164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:28.952740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e853efaf369401581938b17076841bd', 'request_id': 'req-8c36e955-5eea-4296-868d-dc099da48ced'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:28.959845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e853efaf369401581938b17076841bd', 'request_id': 'req-8c36e955-5eea-4296-868d-dc099da48ced'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:28.966793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e853efaf369401581938b17076841bd', 'request_id': 'req-8c36e955-5eea-4296-868d-dc099da48ced'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:28.980344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] The token cache is being invalidate because project 7e853efaf369401581938b17076841bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:28.980759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:28.981299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8c36e955-5eea-4296-868d-dc099da48ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:28.982769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1061/2160] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:28 2022] DELETE /identity/v3/projects/7e853efaf369401581938b17076841bd => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:28.988243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc20fbd-71c7-4b7c-9b2f-6c9201d5a48b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:28.988564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc20fbd-71c7-4b7c-9b2f-6c9201d5a48b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:28.988834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc20fbd-71c7-4b7c-9b2f-6c9201d5a48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.008114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fcc20fbd-71c7-4b7c-9b2f-6c9201d5a48b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:29.009342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fcc20fbd-71c7-4b7c-9b2f-6c9201d5a48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:29.015091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:29.069868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1100/2161] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:29.078543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbddc0-61ae-49d4-a7cc-514b308a3efd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.079483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbddc0-61ae-49d4-a7cc-514b308a3efd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.079761 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbddc0-61ae-49d4-a7cc-514b308a3efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.099660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-94bbddc0-61ae-49d4-a7cc-514b308a3efd None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:29.100945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-94bbddc0-61ae-49d4-a7cc-514b308a3efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:29.109299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1062/2162] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:29.113396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc20fbd-71c7-4b7c-9b2f-6c9201d5a48b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:29.122752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc20fbd-71c7-4b7c-9b2f-6c9201d5a48b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:29.127706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b58ed-878d-4fe4-9d9d-0637abed9146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:29.128289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b58ed-878d-4fe4-9d9d-0637abed9146 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.128599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b58ed-878d-4fe4-9d9d-0637abed9146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.128865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b58ed-878d-4fe4-9d9d-0637abed9146 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.129496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b58ed-878d-4fe4-9d9d-0637abed9146 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:31:29.129790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b58ed-878d-4fe4-9d9d-0637abed9146 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:29.131350 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b58ed-878d-4fe4-9d9d-0637abed9146 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:29.143192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1101/2163] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:31:29.148779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbddc0-61ae-49d4-a7cc-514b308a3efd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:29.157974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbddc0-61ae-49d4-a7cc-514b308a3efd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:29.162903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ba076-3e9b-48de-b0ce-d66ae3bd975b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:29.163489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ba076-3e9b-48de-b0ce-d66ae3bd975b admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.163766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ba076-3e9b-48de-b0ce-d66ae3bd975b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.164035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ba076-3e9b-48de-b0ce-d66ae3bd975b admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.164694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ba076-3e9b-48de-b0ce-d66ae3bd975b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:29.166560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ba076-3e9b-48de-b0ce-d66ae3bd975b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:29.193437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1063/2164] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:29 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:29.197550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b58ed-878d-4fe4-9d9d-0637abed9146 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:29.208036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b58ed-878d-4fe4-9d9d-0637abed9146 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:29.262725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6d764-b721-4369-a0fb-fe05b098ba65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:29.263407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6d764-b721-4369-a0fb-fe05b098ba65 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.263715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6d764-b721-4369-a0fb-fe05b098ba65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.264043 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6d764-b721-4369-a0fb-fe05b098ba65 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.264750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6d764-b721-4369-a0fb-fe05b098ba65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:29.268109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6d764-b721-4369-a0fb-fe05b098ba65 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:29.291812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1102/2165] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:29 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:29.295981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ba076-3e9b-48de-b0ce-d66ae3bd975b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:29.307373 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ba076-3e9b-48de-b0ce-d66ae3bd975b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:29.312109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360e57b-9785-4ea6-94d5-2b93c3f35bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:29.313502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360e57b-9785-4ea6-94d5-2b93c3f35bdc admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.313882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360e57b-9785-4ea6-94d5-2b93c3f35bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.315067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360e57b-9785-4ea6-94d5-2b93c3f35bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.315936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360e57b-9785-4ea6-94d5-2b93c3f35bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:29.317663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360e57b-9785-4ea6-94d5-2b93c3f35bdc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:29.330001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1064/2166] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:29.333857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6d764-b721-4369-a0fb-fe05b098ba65 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:29.343878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6d764-b721-4369-a0fb-fe05b098ba65 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:29.348852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d3b06-6168-4269-a268-b0dad4120942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:29.349443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d3b06-6168-4269-a268-b0dad4120942 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.349714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d3b06-6168-4269-a268-b0dad4120942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.349980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d3b06-6168-4269-a268-b0dad4120942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.350659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d3b06-6168-4269-a268-b0dad4120942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:29.352263 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d3b06-6168-4269-a268-b0dad4120942 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:29.363029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1103/2167] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:29.367031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360e57b-9785-4ea6-94d5-2b93c3f35bdc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:29.373598 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:29.378271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360e57b-9785-4ea6-94d5-2b93c3f35bdc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:29.384256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b2a3-5664-42af-80b2-9c6304ff4ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:29.384948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b2a3-5664-42af-80b2-9c6304ff4ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.385223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b2a3-5664-42af-80b2-9c6304ff4ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.385495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b2a3-5664-42af-80b2-9c6304ff4ab0 admin admin] PATH_INFO: `/v3/projects/20f0cbbd8e1443e1817f9cc71a510097/users/9de00702d61c487d883da563380116cc/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.402603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1b2a3-5664-42af-80b2-9c6304ff4ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20f0cbbd8e1443e1817f9cc71a510097, user_id=9de00702d61c487d883da563380116cc, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:29.405320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1b2a3-5664-42af-80b2-9c6304ff4ab0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:29.418949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1065/2168] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:29 2022] PUT /identity/v3/projects/20f0cbbd8e1443e1817f9cc71a510097/users/9de00702d61c487d883da563380116cc/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:29.429509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07ae30-14ec-4492-b212-01ee54274f19 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.429773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07ae30-14ec-4492-b212-01ee54274f19 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.430040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07ae30-14ec-4492-b212-01ee54274f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.459341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4c07ae30-14ec-4492-b212-01ee54274f19 None None] MFA Rules not processed for user `9de00702d61c487d883da563380116cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:29.460520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4c07ae30-14ec-4492-b212-01ee54274f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:29.522449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1104/2169] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:29.526927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b2a3-5664-42af-80b2-9c6304ff4ab0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:29.586940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b2a3-5664-42af-80b2-9c6304ff4ab0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:29.591585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dda5d0-4421-43f7-ac6f-7812332b9556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:29.592184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dda5d0-4421-43f7-ac6f-7812332b9556 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.592501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dda5d0-4421-43f7-ac6f-7812332b9556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.592793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dda5d0-4421-43f7-ac6f-7812332b9556 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.593395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dda5d0-4421-43f7-ac6f-7812332b9556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:29.595289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dda5d0-4421-43f7-ac6f-7812332b9556 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:29.622744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1066/2170] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:29 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:29.627272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07ae30-14ec-4492-b212-01ee54274f19 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:29.638212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07ae30-14ec-4492-b212-01ee54274f19 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:29.641758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:29.706761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a061402-fbf7-49e1-8cd0-9bd98b414665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:29.707818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a061402-fbf7-49e1-8cd0-9bd98b414665 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.708282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a061402-fbf7-49e1-8cd0-9bd98b414665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.708854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a061402-fbf7-49e1-8cd0-9bd98b414665 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.711785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a061402-fbf7-49e1-8cd0-9bd98b414665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:29.714218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a061402-fbf7-49e1-8cd0-9bd98b414665 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:29.746312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1105/2171] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:29 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:29.747453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dda5d0-4421-43f7-ac6f-7812332b9556 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:29.765653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dda5d0-4421-43f7-ac6f-7812332b9556 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:29.770435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebdd9a-1198-4c8d-b960-2d8547ad6ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:29.771107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebdd9a-1198-4c8d-b960-2d8547ad6ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.771447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebdd9a-1198-4c8d-b960-2d8547ad6ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.771785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebdd9a-1198-4c8d-b960-2d8547ad6ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.772534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebdd9a-1198-4c8d-b960-2d8547ad6ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:29.774189 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebdd9a-1198-4c8d-b960-2d8547ad6ea3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:29.785283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1067/2172] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:29.789262 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a061402-fbf7-49e1-8cd0-9bd98b414665 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:29.801277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a061402-fbf7-49e1-8cd0-9bd98b414665 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:29.806992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff8bf9-5ab5-4905-9668-9983752ce105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:29.807740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff8bf9-5ab5-4905-9668-9983752ce105 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.808035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff8bf9-5ab5-4905-9668-9983752ce105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.808281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff8bf9-5ab5-4905-9668-9983752ce105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.808960 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff8bf9-5ab5-4905-9668-9983752ce105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:29.810562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff8bf9-5ab5-4905-9668-9983752ce105 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:29.821892 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1106/2173] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:29.826075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebdd9a-1198-4c8d-b960-2d8547ad6ea3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:29.836395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebdd9a-1198-4c8d-b960-2d8547ad6ea3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:29.841273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299e060-81c7-4541-89bd-cbed8b380479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:29.841924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299e060-81c7-4541-89bd-cbed8b380479 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.842205 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299e060-81c7-4541-89bd-cbed8b380479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.842482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299e060-81c7-4541-89bd-cbed8b380479 admin admin] PATH_INFO: `/v3/projects/95e5a2d8d6374f3cac4eac6c44d195a9/users/cd2cc84d48e1473bb8de34f0e10b3d5c/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.857813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299e060-81c7-4541-89bd-cbed8b380479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e5a2d8d6374f3cac4eac6c44d195a9, user_id=cd2cc84d48e1473bb8de34f0e10b3d5c, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:29.861102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299e060-81c7-4541-89bd-cbed8b380479 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:29.874308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1068/2174] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:29 2022] PUT /identity/v3/projects/95e5a2d8d6374f3cac4eac6c44d195a9/users/cd2cc84d48e1473bb8de34f0e10b3d5c/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:31:29.884488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d2584-39c1-40f1-aea6-1e3eb795eeb0 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:29.884779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d2584-39c1-40f1-aea6-1e3eb795eeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:29.885082 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d2584-39c1-40f1-aea6-1e3eb795eeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:29.916983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c69d2584-39c1-40f1-aea6-1e3eb795eeb0 None None] MFA Rules not processed for user `cd2cc84d48e1473bb8de34f0e10b3d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:29.918238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c69d2584-39c1-40f1-aea6-1e3eb795eeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:29.981705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1107/2175] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:29.986872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299e060-81c7-4541-89bd-cbed8b380479 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:30.048106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299e060-81c7-4541-89bd-cbed8b380479 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:30.052989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027edfd-77e5-4600-b8c9-c9c2759b3794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:30.053588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027edfd-77e5-4600-b8c9-c9c2759b3794 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:30.053862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027edfd-77e5-4600-b8c9-c9c2759b3794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:30.054130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027edfd-77e5-4600-b8c9-c9c2759b3794 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:30.054783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027edfd-77e5-4600-b8c9-c9c2759b3794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:30.056702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027edfd-77e5-4600-b8c9-c9c2759b3794 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:30.082792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1069/2176] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:29 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:30.086954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d2584-39c1-40f1-aea6-1e3eb795eeb0 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:30.098591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d2584-39c1-40f1-aea6-1e3eb795eeb0 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:30.190705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34daa2-1b4c-45f2-bd4e-c681748169c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:30.192959 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34daa2-1b4c-45f2-bd4e-c681748169c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:30.204799 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34daa2-1b4c-45f2-bd4e-c681748169c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:30.205245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34daa2-1b4c-45f2-bd4e-c681748169c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:30.206269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34daa2-1b4c-45f2-bd4e-c681748169c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:30.208485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34daa2-1b4c-45f2-bd4e-c681748169c1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:30.235821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1108/2177] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:30 2022] POST /identity/v3/users => generated 299 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:30.236967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027edfd-77e5-4600-b8c9-c9c2759b3794 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:30.247354 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027edfd-77e5-4600-b8c9-c9c2759b3794 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:30.254811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1b7e7-982f-427c-817c-60c2ab9cd0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:30.255427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1b7e7-982f-427c-817c-60c2ab9cd0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:30.255712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1b7e7-982f-427c-817c-60c2ab9cd0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:30.255984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1b7e7-982f-427c-817c-60c2ab9cd0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:30.256673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1b7e7-982f-427c-817c-60c2ab9cd0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:30.258290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1b7e7-982f-427c-817c-60c2ab9cd0f2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:30.270147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1070/2178] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:30.275366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34daa2-1b4c-45f2-bd4e-c681748169c1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:30.288698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:30.288698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34daa2-1b4c-45f2-bd4e-c681748169c1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:30.291959 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e2b6a-9326-480c-b91f-39b2d1ed0418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:30.292659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e2b6a-9326-480c-b91f-39b2d1ed0418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:30.292935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e2b6a-9326-480c-b91f-39b2d1ed0418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:30.293204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e2b6a-9326-480c-b91f-39b2d1ed0418 admin admin] PATH_INFO: `/v3/projects/3efca7803e254e93a5e4140787173edf/users/eda424812c464e7dad28a793c85e680a/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:30.308412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e2b6a-9326-480c-b91f-39b2d1ed0418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efca7803e254e93a5e4140787173edf, user_id=eda424812c464e7dad28a793c85e680a, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:30.311051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e2b6a-9326-480c-b91f-39b2d1ed0418 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:30.323515 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1109/2179] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:30 2022] PUT /identity/v3/projects/3efca7803e254e93a5e4140787173edf/users/eda424812c464e7dad28a793c85e680a/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:30.327423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1b7e7-982f-427c-817c-60c2ab9cd0f2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:30.333687 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:30.337422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1b7e7-982f-427c-817c-60c2ab9cd0f2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:30.395569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6af9e6-3c6f-4aa3-bdbf-833ba8259756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:30.396193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af9e6-3c6f-4aa3-bdbf-833ba8259756 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:30.396510 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af9e6-3c6f-4aa3-bdbf-833ba8259756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:30.396790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af9e6-3c6f-4aa3-bdbf-833ba8259756 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:30.397441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6af9e6-3c6f-4aa3-bdbf-833ba8259756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:30.399040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6af9e6-3c6f-4aa3-bdbf-833ba8259756 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:30.410048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1071/2180] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:30.414332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e2b6a-9326-480c-b91f-39b2d1ed0418 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:30.425137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e2b6a-9326-480c-b91f-39b2d1ed0418 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:30.429721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406d931-3f01-4f0d-b41c-94b278583a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:30.430365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406d931-3f01-4f0d-b41c-94b278583a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:30.430644 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406d931-3f01-4f0d-b41c-94b278583a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:30.430942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406d931-3f01-4f0d-b41c-94b278583a33 admin admin] PATH_INFO: `/v3/projects/3efca7803e254e93a5e4140787173edf/users/eda424812c464e7dad28a793c85e680a/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:30.432885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406d931-3f01-4f0d-b41c-94b278583a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efca7803e254e93a5e4140787173edf, user_id=eda424812c464e7dad28a793c85e680a, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:30.435492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406d931-3f01-4f0d-b41c-94b278583a33 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:30.446180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1110/2181] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:30 2022] PUT /identity/v3/projects/3efca7803e254e93a5e4140787173edf/users/eda424812c464e7dad28a793c85e680a/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:30.457152 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70b995-70a7-4d0f-8e45-94bc3d23e771 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:30.457438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70b995-70a7-4d0f-8e45-94bc3d23e771 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:30.457716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70b995-70a7-4d0f-8e45-94bc3d23e771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:30.486623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1b70b995-70a7-4d0f-8e45-94bc3d23e771 None None] MFA Rules not processed for user `eda424812c464e7dad28a793c85e680a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:30.487770 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1b70b995-70a7-4d0f-8e45-94bc3d23e771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:30.557142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1072/2182] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:30.568983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406d931-3f01-4f0d-b41c-94b278583a33 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:30.570920 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b406d931-3f01-4f0d-b41c-94b278583a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:30.627406 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406d931-3f01-4f0d-b41c-94b278583a33 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:30.630948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b4710-aa2d-491a-8758-a4e78fd08f2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:30.631537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b4710-aa2d-491a-8758-a4e78fd08f2b service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:30.631812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b4710-aa2d-491a-8758-a4e78fd08f2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:30.632076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b4710-aa2d-491a-8758-a4e78fd08f2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:30.685916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b4710-aa2d-491a-8758-a4e78fd08f2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:30.687319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b4710-aa2d-491a-8758-a4e78fd08f2b service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:30.696349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1111/2183] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:31:32.264752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70b995-70a7-4d0f-8e45-94bc3d23e771 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:32.266850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1b70b995-70a7-4d0f-8e45-94bc3d23e771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:32.279013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70b995-70a7-4d0f-8e45-94bc3d23e771 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:32.282622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb247f5-eef7-448e-ac7e-c6146d7bbffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:32.283173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb247f5-eef7-448e-ac7e-c6146d7bbffa service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:32.283451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb247f5-eef7-448e-ac7e-c6146d7bbffa service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:32.283733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb247f5-eef7-448e-ac7e-c6146d7bbffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:32.349314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb247f5-eef7-448e-ac7e-c6146d7bbffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:32.350762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb247f5-eef7-448e-ac7e-c6146d7bbffa service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:32.360083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1073/2184] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:33.316269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b4710-aa2d-491a-8758-a4e78fd08f2b service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:33.326017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b4710-aa2d-491a-8758-a4e78fd08f2b service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:33.342028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622aa87a-af4f-473b-95e2-d74b91a1363f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:33.343029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622aa87a-af4f-473b-95e2-d74b91a1363f service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:33.343477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622aa87a-af4f-473b-95e2-d74b91a1363f service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:33.343909 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622aa87a-af4f-473b-95e2-d74b91a1363f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:33.370886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622aa87a-af4f-473b-95e2-d74b91a1363f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:33.372653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622aa87a-af4f-473b-95e2-d74b91a1363f service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:33.398243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1112/2185] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:36.067282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d89c7-0e82-4e84-91f0-ba175a983f11 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:36.067868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d89c7-0e82-4e84-91f0-ba175a983f11 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:36.067868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d89c7-0e82-4e84-91f0-ba175a983f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:36.086311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-644d89c7-0e82-4e84-91f0-ba175a983f11 None None] MFA Rules not processed for user `139dfba82fda43f1a058c60b1a4dc818`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:36.087488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-644d89c7-0e82-4e84-91f0-ba175a983f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:36.105379 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:36.141895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1074/2186] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:42.678975 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622aa87a-af4f-473b-95e2-d74b91a1363f service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:42.746889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622aa87a-af4f-473b-95e2-d74b91a1363f service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:42.752032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:42.752807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:42.753074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:42.753798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] PATH_INFO: `/v3/users/9de00702d61c487d883da563380116cc` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:42.765199 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de00702d61c487d883da563380116cc)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:42.765199 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:42.771003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:42.811864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de00702d61c487d883da563380116cc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:42.818968 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de00702d61c487d883da563380116cc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:42.819708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de00702d61c487d883da563380116cc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:42.828760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de00702d61c487d883da563380116cc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:42.841461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de00702d61c487d883da563380116cc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:42.847962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1113/2187] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:42 2022] DELETE /identity/v3/users/9de00702d61c487d883da563380116cc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:42.852584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adf07e-9d20-42dc-8043-98116e384798 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:42.853106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adf07e-9d20-42dc-8043-98116e384798 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:42.853106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adf07e-9d20-42dc-8043-98116e384798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:42.872034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-78adf07e-9d20-42dc-8043-98116e384798 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:42.873214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-78adf07e-9d20-42dc-8043-98116e384798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:42.930104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1075/2188] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:42.939751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:42.941218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:42.996659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbd8c4-8985-4514-8eab-6db5adf4b5ca admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:43.000206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c264c7-761a-4b93-8ade-20878eac39b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:43.000835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c264c7-761a-4b93-8ade-20878eac39b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:43.001107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c264c7-761a-4b93-8ade-20878eac39b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:43.001370 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c264c7-761a-4b93-8ade-20878eac39b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:43.009934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c264c7-761a-4b93-8ade-20878eac39b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:43.011346 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c264c7-761a-4b93-8ade-20878eac39b0 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:43.021395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1114/2189] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:43.284644 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adf07e-9d20-42dc-8043-98116e384798 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:43.286078 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-78adf07e-9d20-42dc-8043-98116e384798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:43.298823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adf07e-9d20-42dc-8043-98116e384798 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:43.307161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:43.308224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:43.308950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:43.309215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] PATH_INFO: `/v3/projects/20f0cbbd8e1443e1817f9cc71a510097` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:43.310588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] RBAC: Authorizing `identity:delete_project(project_id=20f0cbbd8e1443e1817f9cc71a510097)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:43.313201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:43.368467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20f0cbbd8e1443e1817f9cc71a510097', 'request_id': 'req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:43.374538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20f0cbbd8e1443e1817f9cc71a510097', 'request_id': 'req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:43.383431 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20f0cbbd8e1443e1817f9cc71a510097', 'request_id': 'req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:43.404050 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] The token cache is being invalidate because project 20f0cbbd8e1443e1817f9cc71a510097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:43.404340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:43.405031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:43.406912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1076/2190] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:43 2022] DELETE /identity/v3/projects/20f0cbbd8e1443e1817f9cc71a510097 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:43.410969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c264c7-761a-4b93-8ade-20878eac39b0 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:43.420855 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c264c7-761a-4b93-8ade-20878eac39b0 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:43.496203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:43.496877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:43.497161 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:43.497430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] PATH_INFO: `/v3/users/cd2cc84d48e1473bb8de34f0e10b3d5c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:43.498585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd2cc84d48e1473bb8de34f0e10b3d5c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:43.500533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:43.538022 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:43.551580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd2cc84d48e1473bb8de34f0e10b3d5c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:43.558582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd2cc84d48e1473bb8de34f0e10b3d5c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:43.559299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd2cc84d48e1473bb8de34f0e10b3d5c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:43.566943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd2cc84d48e1473bb8de34f0e10b3d5c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:43.579269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd2cc84d48e1473bb8de34f0e10b3d5c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:43.585743 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1115/2191] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:43 2022] DELETE /identity/v3/users/cd2cc84d48e1473bb8de34f0e10b3d5c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:43.860944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:43.863204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:43.878242 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd7398-69e3-4c9d-9d8d-78c51db1f12b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:43.927876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:43.937523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:43.938608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:43.938840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:43.939134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] PATH_INFO: `/v3/projects/95e5a2d8d6374f3cac4eac6c44d195a9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:43.941064 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95e5a2d8d6374f3cac4eac6c44d195a9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:43.943220 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:43.998838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e5a2d8d6374f3cac4eac6c44d195a9', 'request_id': 'req-e643e6ce-298d-4ced-9840-a47ff7955053'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.005156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e5a2d8d6374f3cac4eac6c44d195a9', 'request_id': 'req-e643e6ce-298d-4ced-9840-a47ff7955053'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.017622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e5a2d8d6374f3cac4eac6c44d195a9', 'request_id': 'req-e643e6ce-298d-4ced-9840-a47ff7955053'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.031164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] The token cache is being invalidate because project 95e5a2d8d6374f3cac4eac6c44d195a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:44.031456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.032227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.033796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1077/2192] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:43 2022] DELETE /identity/v3/projects/95e5a2d8d6374f3cac4eac6c44d195a9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:44.038483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:44.039962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:44.053026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eac1cc-f4cb-4c42-9eb1-21aa523ab562 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:44.114580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135607d-5eb0-476f-ba58-00eee93c0543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:44.115215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135607d-5eb0-476f-ba58-00eee93c0543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:44.115488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135607d-5eb0-476f-ba58-00eee93c0543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:44.115761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135607d-5eb0-476f-ba58-00eee93c0543 admin admin] PATH_INFO: `/v3/users/eda424812c464e7dad28a793c85e680a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:44.116918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135607d-5eb0-476f-ba58-00eee93c0543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda424812c464e7dad28a793c85e680a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:44.119005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135607d-5eb0-476f-ba58-00eee93c0543 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:44.170804 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b135607d-5eb0-476f-ba58-00eee93c0543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda424812c464e7dad28a793c85e680a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.179483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b135607d-5eb0-476f-ba58-00eee93c0543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda424812c464e7dad28a793c85e680a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.180520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b135607d-5eb0-476f-ba58-00eee93c0543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda424812c464e7dad28a793c85e680a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.188872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b135607d-5eb0-476f-ba58-00eee93c0543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda424812c464e7dad28a793c85e680a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.201249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b135607d-5eb0-476f-ba58-00eee93c0543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda424812c464e7dad28a793c85e680a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.207092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1116/2193] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:44 2022] DELETE /identity/v3/users/eda424812c464e7dad28a793c85e680a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:44.508121 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:44.509512 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:44.523225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643e6ce-298d-4ced-9840-a47ff7955053 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:44.622186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:44.622902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:44.623554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:44.623554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] PATH_INFO: `/v3/projects/3efca7803e254e93a5e4140787173edf` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:44.625064 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3efca7803e254e93a5e4140787173edf)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:44.627463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:44.696183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3efca7803e254e93a5e4140787173edf', 'request_id': 'req-129ee983-2d90-40ef-97ea-9fd50334307e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.697272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3efca7803e254e93a5e4140787173edf', 'request_id': 'req-129ee983-2d90-40ef-97ea-9fd50334307e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.703862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3efca7803e254e93a5e4140787173edf', 'request_id': 'req-129ee983-2d90-40ef-97ea-9fd50334307e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.720032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] The token cache is being invalidate because project 3efca7803e254e93a5e4140787173edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:44.720307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.720964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-129ee983-2d90-40ef-97ea-9fd50334307e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:44.727116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1078/2194] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:44 2022] DELETE /identity/v3/projects/3efca7803e254e93a5e4140787173edf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:44.729829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd0eaa-8157-448a-abf4-b1d399348006 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:44.730244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd0eaa-8157-448a-abf4-b1d399348006 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:44.730529 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd0eaa-8157-448a-abf4-b1d399348006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:44.760082 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ebfd0eaa-8157-448a-abf4-b1d399348006 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:44.761486 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ebfd0eaa-8157-448a-abf4-b1d399348006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:44.791752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:44.849408 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1117/2195] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:44.853245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34505c7-6f68-4abc-82e8-4327d985dd08 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:44.853631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34505c7-6f68-4abc-82e8-4327d985dd08 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:44.854004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34505c7-6f68-4abc-82e8-4327d985dd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:44.878820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c34505c7-6f68-4abc-82e8-4327d985dd08 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:44.880085 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c34505c7-6f68-4abc-82e8-4327d985dd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:44.888353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1079/2196] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:44.892601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd0eaa-8157-448a-abf4-b1d399348006 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:44.904551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd0eaa-8157-448a-abf4-b1d399348006 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:44.911137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7997a-e07d-4215-be2d-cd94a1b14cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:44.911792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7997a-e07d-4215-be2d-cd94a1b14cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:44.912069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7997a-e07d-4215-be2d-cd94a1b14cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:44.912339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7997a-e07d-4215-be2d-cd94a1b14cc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:44.913023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7997a-e07d-4215-be2d-cd94a1b14cc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:31:44.913324 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7997a-e07d-4215-be2d-cd94a1b14cc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:44.915032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7997a-e07d-4215-be2d-cd94a1b14cc5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:44.931720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1118/2197] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:31:44.933370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34505c7-6f68-4abc-82e8-4327d985dd08 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:44.945635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34505c7-6f68-4abc-82e8-4327d985dd08 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:44.951272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c657b18-b2b7-4506-92ab-10a0daa50ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:44.952236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c657b18-b2b7-4506-92ab-10a0daa50ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:44.952553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c657b18-b2b7-4506-92ab-10a0daa50ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:44.952826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c657b18-b2b7-4506-92ab-10a0daa50ba7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:44.953483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c657b18-b2b7-4506-92ab-10a0daa50ba7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:44.955519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c657b18-b2b7-4506-92ab-10a0daa50ba7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:44.961731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:45.003130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1080/2198] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:44 2022] POST /identity/v3/projects => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:45.004187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7997a-e07d-4215-be2d-cd94a1b14cc5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:45.014317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7997a-e07d-4215-be2d-cd94a1b14cc5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:45.077768 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec85fde-6afc-4568-a0d1-3c6141562698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:45.078471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec85fde-6afc-4568-a0d1-3c6141562698 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:45.078966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec85fde-6afc-4568-a0d1-3c6141562698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:45.079307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec85fde-6afc-4568-a0d1-3c6141562698 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:45.080065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec85fde-6afc-4568-a0d1-3c6141562698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:45.082179 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec85fde-6afc-4568-a0d1-3c6141562698 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:45.109106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1119/2199] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:45 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:45.110381 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c657b18-b2b7-4506-92ab-10a0daa50ba7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:45.122486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c657b18-b2b7-4506-92ab-10a0daa50ba7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:45.128486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37eaa7-acce-43ff-ae7c-e65c284544b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:45.129329 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37eaa7-acce-43ff-ae7c-e65c284544b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:45.131577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37eaa7-acce-43ff-ae7c-e65c284544b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:45.131577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37eaa7-acce-43ff-ae7c-e65c284544b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:45.131577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37eaa7-acce-43ff-ae7c-e65c284544b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:45.153372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37eaa7-acce-43ff-ae7c-e65c284544b4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:45.168221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1081/2200] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:45.171397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec85fde-6afc-4568-a0d1-3c6141562698 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:45.182094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec85fde-6afc-4568-a0d1-3c6141562698 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:45.187013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa616e-bb5f-493a-b19e-70c0f08f07fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:45.187605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa616e-bb5f-493a-b19e-70c0f08f07fc admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:45.187876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa616e-bb5f-493a-b19e-70c0f08f07fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:45.188140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa616e-bb5f-493a-b19e-70c0f08f07fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:45.188810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa616e-bb5f-493a-b19e-70c0f08f07fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:45.190409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa616e-bb5f-493a-b19e-70c0f08f07fc admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:45.200876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1120/2201] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:45.205241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37eaa7-acce-43ff-ae7c-e65c284544b4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:45.215850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37eaa7-acce-43ff-ae7c-e65c284544b4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:45.220596 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65954a82-20a3-423e-9c6a-1e1a8dd08b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:45.222032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65954a82-20a3-423e-9c6a-1e1a8dd08b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:45.222385 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65954a82-20a3-423e-9c6a-1e1a8dd08b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:45.222720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65954a82-20a3-423e-9c6a-1e1a8dd08b91 admin admin] PATH_INFO: `/v3/projects/c6a016a9695c47e18d335d633adf15a0/users/24628c74de5b413cb493c328bf05866a/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:45.240930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65954a82-20a3-423e-9c6a-1e1a8dd08b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a016a9695c47e18d335d633adf15a0, user_id=24628c74de5b413cb493c328bf05866a, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:45.243627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65954a82-20a3-423e-9c6a-1e1a8dd08b91 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:45.256479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1082/2202] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:45 2022] PUT /identity/v3/projects/c6a016a9695c47e18d335d633adf15a0/users/24628c74de5b413cb493c328bf05866a/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:45.323702 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f07360-96c8-4432-b33a-4b6da2fdb07c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:45.324032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f07360-96c8-4432-b33a-4b6da2fdb07c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:45.324245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f07360-96c8-4432-b33a-4b6da2fdb07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:45.352393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-58f07360-96c8-4432-b33a-4b6da2fdb07c None None] MFA Rules not processed for user `24628c74de5b413cb493c328bf05866a`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:45.353602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-58f07360-96c8-4432-b33a-4b6da2fdb07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:45.416279 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1121/2203] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:45.427634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65954a82-20a3-423e-9c6a-1e1a8dd08b91 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:45.429317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-65954a82-20a3-423e-9c6a-1e1a8dd08b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:45.483906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65954a82-20a3-423e-9c6a-1e1a8dd08b91 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:45.487522 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c394cc4-c3e0-4bf9-971c-f51c4a4555f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a9b371434243daa4176215daefb79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:45.488125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c394cc4-c3e0-4bf9-971c-f51c4a4555f0 service glance] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:45.488422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c394cc4-c3e0-4bf9-971c-f51c4a4555f0 service glance] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:45.488697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c394cc4-c3e0-4bf9-971c-f51c4a4555f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:45.496774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c394cc4-c3e0-4bf9-971c-f51c4a4555f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:45.498179 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c394cc4-c3e0-4bf9-971c-f51c4a4555f0 service glance] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:45.507203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1083/2204] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:46.123633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f07360-96c8-4432-b33a-4b6da2fdb07c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:46.125074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-58f07360-96c8-4432-b33a-4b6da2fdb07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:46.152493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:46.193332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f07360-96c8-4432-b33a-4b6da2fdb07c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:46.198106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:46.198867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:46.199147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:46.199415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] PATH_INFO: `/v3/users/0f4f9dce871844d0aa83ba2a2ba88274` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:46.200576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f4f9dce871844d0aa83ba2a2ba88274)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:46.202467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:46.259652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f4f9dce871844d0aa83ba2a2ba88274'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:46.267037 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f4f9dce871844d0aa83ba2a2ba88274'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:46.267708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f4f9dce871844d0aa83ba2a2ba88274'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:46.275770 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f4f9dce871844d0aa83ba2a2ba88274'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:46.288637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f4f9dce871844d0aa83ba2a2ba88274'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:46.294728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1122/2205] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:46 2022] DELETE /identity/v3/users/0f4f9dce871844d0aa83ba2a2ba88274 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:46.299098 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51cd9-2115-4e54-bf6a-4c36e4f05722 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:46.299607 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51cd9-2115-4e54-bf6a-4c36e4f05722 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:46.299702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51cd9-2115-4e54-bf6a-4c36e4f05722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:46.319230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-14b51cd9-2115-4e54-bf6a-4c36e4f05722 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:46.320544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-14b51cd9-2115-4e54-bf6a-4c36e4f05722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:46.381936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1084/2206] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:31:46.395551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:46.397254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:46.457854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7c0e5-e45b-4bfe-b47d-4d4f2ac56ee0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:46.461529 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa621f-4043-4f1b-bfb1-06428d1c14bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:46.462136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa621f-4043-4f1b-bfb1-06428d1c14bc service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:46.462407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa621f-4043-4f1b-bfb1-06428d1c14bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:46.462673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa621f-4043-4f1b-bfb1-06428d1c14bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:46.471764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa621f-4043-4f1b-bfb1-06428d1c14bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:46.473215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa621f-4043-4f1b-bfb1-06428d1c14bc service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:46.482798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1123/2207] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:46.762988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b51cd9-2115-4e54-bf6a-4c36e4f05722 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:46.765527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-14b51cd9-2115-4e54-bf6a-4c36e4f05722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:46.790278 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b51cd9-2115-4e54-bf6a-4c36e4f05722 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:46.797346 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:46.797970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:46.798245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:46.798515 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] PATH_INFO: `/v3/projects/8966e68dee3a4f73b1d4d4d8c748ecec` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:46.800158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8966e68dee3a4f73b1d4d4d8c748ecec)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:46.802099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:46.848988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:46.850445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8966e68dee3a4f73b1d4d4d8c748ecec', 'request_id': 'req-a8029a99-a444-4fcf-a7e8-b30bed72ed48'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:46.857339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8966e68dee3a4f73b1d4d4d8c748ecec', 'request_id': 'req-a8029a99-a444-4fcf-a7e8-b30bed72ed48'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:46.864230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8966e68dee3a4f73b1d4d4d8c748ecec', 'request_id': 'req-a8029a99-a444-4fcf-a7e8-b30bed72ed48'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:46.875896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] The token cache is being invalidate because project 8966e68dee3a4f73b1d4d4d8c748ecec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:46.876179 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:46.876826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a8029a99-a444-4fcf-a7e8-b30bed72ed48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:46.878949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1085/2208] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:46 2022] DELETE /identity/v3/projects/8966e68dee3a4f73b1d4d4d8c748ecec => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:46.883776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4f1a3-a6dd-403e-b302-c33ead87394a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:46.884054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4f1a3-a6dd-403e-b302-c33ead87394a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:46.884318 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4f1a3-a6dd-403e-b302-c33ead87394a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:46.903741 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-20c4f1a3-a6dd-403e-b302-c33ead87394a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:46.904534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-20c4f1a3-a6dd-403e-b302-c33ead87394a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:46.950739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:46.970829 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1124/2209] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:31:46.978492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438deea-458d-4c4a-a5a4-f8b87c1586da None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:46.978782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438deea-458d-4c4a-a5a4-f8b87c1586da None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:46.979058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438deea-458d-4c4a-a5a4-f8b87c1586da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:47.001058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e438deea-458d-4c4a-a5a4-f8b87c1586da None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:47.002279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e438deea-458d-4c4a-a5a4-f8b87c1586da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:47.012523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1086/2210] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:47.016842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4f1a3-a6dd-403e-b302-c33ead87394a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:47.027642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4f1a3-a6dd-403e-b302-c33ead87394a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:47.032753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427135e7-2454-4f33-9f85-cc06fd5de80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:47.033356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427135e7-2454-4f33-9f85-cc06fd5de80e admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:47.033667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427135e7-2454-4f33-9f85-cc06fd5de80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:47.034653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427135e7-2454-4f33-9f85-cc06fd5de80e admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:47.039746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427135e7-2454-4f33-9f85-cc06fd5de80e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:31:47.040068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427135e7-2454-4f33-9f85-cc06fd5de80e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:47.041699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427135e7-2454-4f33-9f85-cc06fd5de80e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:47.054910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1125/2211] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:31:47.059664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438deea-458d-4c4a-a5a4-f8b87c1586da None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:47.069522 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438deea-458d-4c4a-a5a4-f8b87c1586da None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:47.074860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b4408-10f8-48a8-805f-359b46754155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:47.076002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b4408-10f8-48a8-805f-359b46754155 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:47.076686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b4408-10f8-48a8-805f-359b46754155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:47.077152 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b4408-10f8-48a8-805f-359b46754155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:47.078133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b4408-10f8-48a8-805f-359b46754155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:47.080618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b4408-10f8-48a8-805f-359b46754155 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:47.115668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1087/2212] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:47 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:47.119899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427135e7-2454-4f33-9f85-cc06fd5de80e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:47.131070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427135e7-2454-4f33-9f85-cc06fd5de80e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:47.191800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64780005-a8fa-4718-999a-4a89225efe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:47.192448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64780005-a8fa-4718-999a-4a89225efe2d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:47.192725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64780005-a8fa-4718-999a-4a89225efe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:47.192993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64780005-a8fa-4718-999a-4a89225efe2d admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:47.193619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64780005-a8fa-4718-999a-4a89225efe2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:47.195551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64780005-a8fa-4718-999a-4a89225efe2d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:47.220642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1126/2213] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:47 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:47.222461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b4408-10f8-48a8-805f-359b46754155 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:47.233710 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b4408-10f8-48a8-805f-359b46754155 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:47.238924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7367ba-4f42-4310-b0c5-92276db186f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:47.239537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7367ba-4f42-4310-b0c5-92276db186f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:47.239810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7367ba-4f42-4310-b0c5-92276db186f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:47.240076 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7367ba-4f42-4310-b0c5-92276db186f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:47.240765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7367ba-4f42-4310-b0c5-92276db186f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:47.242370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7367ba-4f42-4310-b0c5-92276db186f3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:47.252727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1088/2214] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:47.256658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64780005-a8fa-4718-999a-4a89225efe2d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:47.266674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64780005-a8fa-4718-999a-4a89225efe2d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:47.271369 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e23f4-8cf5-4b9b-99c8-c2fedce952e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:47.271996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e23f4-8cf5-4b9b-99c8-c2fedce952e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:47.272272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e23f4-8cf5-4b9b-99c8-c2fedce952e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:47.272579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e23f4-8cf5-4b9b-99c8-c2fedce952e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:47.273222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e23f4-8cf5-4b9b-99c8-c2fedce952e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:47.274889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e23f4-8cf5-4b9b-99c8-c2fedce952e7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:47.286161 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1127/2215] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:47.290696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7367ba-4f42-4310-b0c5-92276db186f3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:47.301688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7367ba-4f42-4310-b0c5-92276db186f3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:47.306450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16967f92-3066-4ecc-a881-1fc1e623af25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:47.307103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16967f92-3066-4ecc-a881-1fc1e623af25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:47.307384 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16967f92-3066-4ecc-a881-1fc1e623af25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:47.307657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16967f92-3066-4ecc-a881-1fc1e623af25 admin admin] PATH_INFO: `/v3/projects/3ce0ef6ddaba4abd91781a620729b40c/users/5169fcc873404a1ca0f199f8af35f7b3/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:47.322209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16967f92-3066-4ecc-a881-1fc1e623af25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce0ef6ddaba4abd91781a620729b40c, user_id=5169fcc873404a1ca0f199f8af35f7b3, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:47.324914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16967f92-3066-4ecc-a881-1fc1e623af25 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:47.338375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1089/2216] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:47 2022] PUT /identity/v3/projects/3ce0ef6ddaba4abd91781a620729b40c/users/5169fcc873404a1ca0f199f8af35f7b3/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:47.350477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79735e-7e2d-4545-b066-58585303cf66 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:47.350766 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79735e-7e2d-4545-b066-58585303cf66 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:47.351040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79735e-7e2d-4545-b066-58585303cf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:47.380927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0e79735e-7e2d-4545-b066-58585303cf66 None None] MFA Rules not processed for user `5169fcc873404a1ca0f199f8af35f7b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:47.382154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0e79735e-7e2d-4545-b066-58585303cf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:47.459090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1128/2217] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:47.470791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16967f92-3066-4ecc-a881-1fc1e623af25 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:47.473409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-16967f92-3066-4ecc-a881-1fc1e623af25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:47.535040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16967f92-3066-4ecc-a881-1fc1e623af25 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:47.538615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1e094-cfe6-4f4f-92e8-65723607df95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:47.539221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1e094-cfe6-4f4f-92e8-65723607df95 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:47.539500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1e094-cfe6-4f4f-92e8-65723607df95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:47.539775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1e094-cfe6-4f4f-92e8-65723607df95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:47.548171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1e094-cfe6-4f4f-92e8-65723607df95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:47.549676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1e094-cfe6-4f4f-92e8-65723607df95 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:47.558332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1090/2218] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:50.136005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79735e-7e2d-4545-b066-58585303cf66 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:50.206148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:50.228105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79735e-7e2d-4545-b066-58585303cf66 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:50.232945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:50.233697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:50.233979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:50.234248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] PATH_INFO: `/v3/users/5169fcc873404a1ca0f199f8af35f7b3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:50.235369 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5169fcc873404a1ca0f199f8af35f7b3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:50.238000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:50.286565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5169fcc873404a1ca0f199f8af35f7b3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:50.293314 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5169fcc873404a1ca0f199f8af35f7b3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:50.293985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5169fcc873404a1ca0f199f8af35f7b3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:50.302972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5169fcc873404a1ca0f199f8af35f7b3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:50.315276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5169fcc873404a1ca0f199f8af35f7b3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:50.321101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1129/2219] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:31:50 2022] DELETE /identity/v3/users/5169fcc873404a1ca0f199f8af35f7b3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:50.325956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f3e08-8713-4247-a780-9ebae16231a0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:50.326429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f3e08-8713-4247-a780-9ebae16231a0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:50.326635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f3e08-8713-4247-a780-9ebae16231a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:50.345370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2a7f3e08-8713-4247-a780-9ebae16231a0 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:50.347162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2a7f3e08-8713-4247-a780-9ebae16231a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:50.406149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1091/2220] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:31:50.416934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:50.418351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:50.479737 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adf478-d820-49e1-8b2d-896696dc5cc0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:50.483436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0995fe-5a0b-4dee-9a44-8017cf910f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:50.484038 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0995fe-5a0b-4dee-9a44-8017cf910f2c service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:50.484335 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0995fe-5a0b-4dee-9a44-8017cf910f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:50.484640 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0995fe-5a0b-4dee-9a44-8017cf910f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:50.493819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0995fe-5a0b-4dee-9a44-8017cf910f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:50.495294 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0995fe-5a0b-4dee-9a44-8017cf910f2c service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:50.505373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1130/2221] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:50.785200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f3e08-8713-4247-a780-9ebae16231a0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:50.786756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2a7f3e08-8713-4247-a780-9ebae16231a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:50.800898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f3e08-8713-4247-a780-9ebae16231a0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:50.806520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:50.807107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:50.807488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:50.807865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] PATH_INFO: `/v3/projects/3ce0ef6ddaba4abd91781a620729b40c` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:50.809168 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce0ef6ddaba4abd91781a620729b40c)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:50.811202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:50.828679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:50.867338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce0ef6ddaba4abd91781a620729b40c', 'request_id': 'req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:50.872958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce0ef6ddaba4abd91781a620729b40c', 'request_id': 'req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:50.879785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce0ef6ddaba4abd91781a620729b40c', 'request_id': 'req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:50.893056 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] The token cache is being invalidate because project 3ce0ef6ddaba4abd91781a620729b40c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:31:50.893997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:50.894655 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5155b0fd-0855-4acd-a0c5-08b2f5eb11e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:31:50.896204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1092/2222] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:31:50 2022] DELETE /identity/v3/projects/3ce0ef6ddaba4abd91781a620729b40c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:50.901246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e0cd1-96b1-443b-a76b-7cba8fae433e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:50.902280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e0cd1-96b1-443b-a76b-7cba8fae433e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:50.902558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e0cd1-96b1-443b-a76b-7cba8fae433e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:50.923708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1d5e0cd1-96b1-443b-a76b-7cba8fae433e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:50.925119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1d5e0cd1-96b1-443b-a76b-7cba8fae433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:51.023526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1131/2223] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:31:51.031849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6baed-f91c-411a-9a82-171cabf0e90e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:51.032290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6baed-f91c-411a-9a82-171cabf0e90e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:51.032729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6baed-f91c-411a-9a82-171cabf0e90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:51.054045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-56a6baed-f91c-411a-9a82-171cabf0e90e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:51.055273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-56a6baed-f91c-411a-9a82-171cabf0e90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:51.064888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1093/2224] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:51.069917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e0cd1-96b1-443b-a76b-7cba8fae433e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:51.075901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:51.079723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e0cd1-96b1-443b-a76b-7cba8fae433e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:51.087424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce883f6-db35-4b47-aac8-bfba56f2fe08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:51.088440 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce883f6-db35-4b47-aac8-bfba56f2fe08 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:51.088756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce883f6-db35-4b47-aac8-bfba56f2fe08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:51.089103 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce883f6-db35-4b47-aac8-bfba56f2fe08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:51.089923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce883f6-db35-4b47-aac8-bfba56f2fe08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:31:51.090284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce883f6-db35-4b47-aac8-bfba56f2fe08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:51.091987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce883f6-db35-4b47-aac8-bfba56f2fe08 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:51.103958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1132/2225] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:31:51.108390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6baed-f91c-411a-9a82-171cabf0e90e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:51.118645 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6baed-f91c-411a-9a82-171cabf0e90e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:51.123235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d8e2d-6881-4d49-9f00-195cf375dd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:51.123829 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d8e2d-6881-4d49-9f00-195cf375dd00 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:51.124107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d8e2d-6881-4d49-9f00-195cf375dd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:51.124400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d8e2d-6881-4d49-9f00-195cf375dd00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:51.125039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d8e2d-6881-4d49-9f00-195cf375dd00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:51.126934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d8e2d-6881-4d49-9f00-195cf375dd00 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:51.157478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1094/2226] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:31:51 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:51.159493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce883f6-db35-4b47-aac8-bfba56f2fe08 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:51.169481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce883f6-db35-4b47-aac8-bfba56f2fe08 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:51.225701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e3956-1d47-4213-a9a6-1a7f3e163609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:51.226324 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e3956-1d47-4213-a9a6-1a7f3e163609 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:51.226600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e3956-1d47-4213-a9a6-1a7f3e163609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:51.226867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e3956-1d47-4213-a9a6-1a7f3e163609 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:51.227512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e3956-1d47-4213-a9a6-1a7f3e163609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:51.229484 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e3956-1d47-4213-a9a6-1a7f3e163609 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:51.256432 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1133/2227] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:31:51 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:31:51.257694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d8e2d-6881-4d49-9f00-195cf375dd00 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:51.268707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d8e2d-6881-4d49-9f00-195cf375dd00 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:51.273500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07163fd1-ffa9-4830-a4db-a9319fe0bcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:51.274153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163fd1-ffa9-4830-a4db-a9319fe0bcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:51.274436 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163fd1-ffa9-4830-a4db-a9319fe0bcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:51.274725 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163fd1-ffa9-4830-a4db-a9319fe0bcd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:51.275358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07163fd1-ffa9-4830-a4db-a9319fe0bcd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:51.277008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07163fd1-ffa9-4830-a4db-a9319fe0bcd3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:51.288143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1095/2228] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:51.292038 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e3956-1d47-4213-a9a6-1a7f3e163609 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:51.302425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e3956-1d47-4213-a9a6-1a7f3e163609 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:51.307116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d7ce2-8e7c-42ea-9ce1-bfb022a915aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:51.307711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d7ce2-8e7c-42ea-9ce1-bfb022a915aa admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:51.307994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d7ce2-8e7c-42ea-9ce1-bfb022a915aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:51.308268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d7ce2-8e7c-42ea-9ce1-bfb022a915aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:51.308955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d7ce2-8e7c-42ea-9ce1-bfb022a915aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:51.310566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d7ce2-8e7c-42ea-9ce1-bfb022a915aa admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:51.322044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1134/2229] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:31:51.326232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07163fd1-ffa9-4830-a4db-a9319fe0bcd3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:51.336175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07163fd1-ffa9-4830-a4db-a9319fe0bcd3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:51.341138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d589b-38ee-4a58-8d41-dbf97203704c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:51.341779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d589b-38ee-4a58-8d41-dbf97203704c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:51.342051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d589b-38ee-4a58-8d41-dbf97203704c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:51.342317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d589b-38ee-4a58-8d41-dbf97203704c admin admin] PATH_INFO: `/v3/projects/3c3d05188b0747a2920a3e9101d5828d/users/ab8c2d9d39574d8ca94114b7eee59f16/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:51.356959 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d589b-38ee-4a58-8d41-dbf97203704c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c3d05188b0747a2920a3e9101d5828d, user_id=ab8c2d9d39574d8ca94114b7eee59f16, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:51.359583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d589b-38ee-4a58-8d41-dbf97203704c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:51.373752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1096/2230] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:31:51 2022] PUT /identity/v3/projects/3c3d05188b0747a2920a3e9101d5828d/users/ab8c2d9d39574d8ca94114b7eee59f16/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:31:51.384994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687d07c-1d02-410c-ac20-ca749889ea12 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:51.385389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687d07c-1d02-410c-ac20-ca749889ea12 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:51.385716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687d07c-1d02-410c-ac20-ca749889ea12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:51.414973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-6687d07c-1d02-410c-ac20-ca749889ea12 None None] MFA Rules not processed for user `ab8c2d9d39574d8ca94114b7eee59f16`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:31:51.416183 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6687d07c-1d02-410c-ac20-ca749889ea12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:51.480513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1135/2231] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:31:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:31:51.493042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d589b-38ee-4a58-8d41-dbf97203704c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:51.494704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ba4d589b-38ee-4a58-8d41-dbf97203704c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:51.552518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d589b-38ee-4a58-8d41-dbf97203704c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:51.556186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a0060-4562-49d8-b838-20a854d66bf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:51.556976 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a0060-4562-49d8-b838-20a854d66bf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:51.557316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a0060-4562-49d8-b838-20a854d66bf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:51.557599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a0060-4562-49d8-b838-20a854d66bf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:51.566387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a0060-4562-49d8-b838-20a854d66bf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:51.567825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a0060-4562-49d8-b838-20a854d66bf7 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:51.577253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1097/2232] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:31:54.602485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687d07c-1d02-410c-ac20-ca749889ea12 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:31:54.604194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6687d07c-1d02-410c-ac20-ca749889ea12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:54.661944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687d07c-1d02-410c-ac20-ca749889ea12 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:31:54.665685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591e3a3-83e0-4217-ae3a-4567356e22c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:31:54.666286 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591e3a3-83e0-4217-ae3a-4567356e22c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:31:54.666565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591e3a3-83e0-4217-ae3a-4567356e22c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:31:54.666832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591e3a3-83e0-4217-ae3a-4567356e22c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:31:54.669114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7591e3a3-83e0-4217-ae3a-4567356e22c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:31:54.684209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:31:54.740161 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591e3a3-83e0-4217-ae3a-4567356e22c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:31:54.741782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591e3a3-83e0-4217-ae3a-4567356e22c6 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:31:54.754330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1136/2233] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:31:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:32:01.572991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:32:04.747636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=23 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:32:14.548308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e717-25f4-499f-99bf-6fa4edd1e458 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:14.548937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e717-25f4-499f-99bf-6fa4edd1e458 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:14.548937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e717-25f4-499f-99bf-6fa4edd1e458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:14.568461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7e82e717-25f4-499f-99bf-6fa4edd1e458 None None] MFA Rules not processed for user `624738daa9db4aff885d71ea2d65fb26`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:32:14.569651 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7e82e717-25f4-499f-99bf-6fa4edd1e458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:14.639459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1098/2234] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:32:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:32:15.732763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591e3a3-83e0-4217-ae3a-4567356e22c6 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:15.734203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7591e3a3-83e0-4217-ae3a-4567356e22c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:15.801858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591e3a3-83e0-4217-ae3a-4567356e22c6 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:15.806694 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70b8dc-ded1-4c23-b310-7115f55203a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:15.807287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70b8dc-ded1-4c23-b310-7115f55203a1 service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:15.807563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70b8dc-ded1-4c23-b310-7115f55203a1 service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:15.807828 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70b8dc-ded1-4c23-b310-7115f55203a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:15.809539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dc70b8dc-ded1-4c23-b310-7115f55203a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:15.821042 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70b8dc-ded1-4c23-b310-7115f55203a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:15.822592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70b8dc-ded1-4c23-b310-7115f55203a1 service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:15.832076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1137/2235] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:32:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:32:22.639489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674586fa-1015-490b-a948-4bac55258533 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:22.640423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674586fa-1015-490b-a948-4bac55258533 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:22.640975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674586fa-1015-490b-a948-4bac55258533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:22.663905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-674586fa-1015-490b-a948-4bac55258533 None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:32:22.666015 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-674586fa-1015-490b-a948-4bac55258533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:22.724962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1099/2236] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:32:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:32:22.765431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70b8dc-ded1-4c23-b310-7115f55203a1 service placement] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:22.767607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dc70b8dc-ded1-4c23-b310-7115f55203a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:22.842626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70b8dc-ded1-4c23-b310-7115f55203a1 service placement] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:22.847642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3978ac1-5251-4bfe-9f36-84085531b234 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:22.848247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3978ac1-5251-4bfe-9f36-84085531b234 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:22.848579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3978ac1-5251-4bfe-9f36-84085531b234 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:22.848850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3978ac1-5251-4bfe-9f36-84085531b234 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:22.856801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3978ac1-5251-4bfe-9f36-84085531b234 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:22.858479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3978ac1-5251-4bfe-9f36-84085531b234 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:22.867164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1138/2237] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:32:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:32:32.716802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=50 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:32:32.863111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=64 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:32:47.468154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc951fc2-1c70-4ee6-bda6-ec4c4a345bd5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:47.468797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc951fc2-1c70-4ee6-bda6-ec4c4a345bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:47.469017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc951fc2-1c70-4ee6-bda6-ec4c4a345bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:47.493956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-fc951fc2-1c70-4ee6-bda6-ec4c4a345bd5 None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:32:47.496919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fc951fc2-1c70-4ee6-bda6-ec4c4a345bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:47.511468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1100/2238] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 30 19:32:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:32:47.519601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3978ac1-5251-4bfe-9f36-84085531b234 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:47.533705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3978ac1-5251-4bfe-9f36-84085531b234 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:47.545115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae3723-a52b-4c08-a651-7f8bdc704759 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:47.545732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae3723-a52b-4c08-a651-7f8bdc704759 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:47.546006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae3723-a52b-4c08-a651-7f8bdc704759 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:47.546272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae3723-a52b-4c08-a651-7f8bdc704759 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:47.563658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae3723-a52b-4c08-a651-7f8bdc704759 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:47.565292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae3723-a52b-4c08-a651-7f8bdc704759 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:47.576304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1139/2239] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:32:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:32:49.602212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc951fc2-1c70-4ee6-bda6-ec4c4a345bd5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:49.603740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fc951fc2-1c70-4ee6-bda6-ec4c4a345bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:49.674716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc951fc2-1c70-4ee6-bda6-ec4c4a345bd5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:49.679799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:49.680467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:49.680805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:49.681107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] PATH_INFO: `/v3/users/139dfba82fda43f1a058c60b1a4dc818` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:49.683334 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=139dfba82fda43f1a058c60b1a4dc818)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:49.685271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:49.732412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '139dfba82fda43f1a058c60b1a4dc818'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:49.740658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '139dfba82fda43f1a058c60b1a4dc818'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:49.741404 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '139dfba82fda43f1a058c60b1a4dc818'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:49.748737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '139dfba82fda43f1a058c60b1a4dc818'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:49.760823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '139dfba82fda43f1a058c60b1a4dc818'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:49.766510 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1101/2240] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:32:49 2022] DELETE /identity/v3/users/139dfba82fda43f1a058c60b1a4dc818 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:32:49.954241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae3723-a52b-4c08-a651-7f8bdc704759 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:49.966444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae3723-a52b-4c08-a651-7f8bdc704759 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:50.023589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:50.024126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:50.024431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:50.024725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] PATH_INFO: `/v3/projects/08e6fee821e7449b92c3df24de169fbc` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:50.025830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e6fee821e7449b92c3df24de169fbc)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:50.027705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:50.073453 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e6fee821e7449b92c3df24de169fbc', 'request_id': 'req-6659cdd3-e942-45a3-9736-c068ee4690ee'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:50.079071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e6fee821e7449b92c3df24de169fbc', 'request_id': 'req-6659cdd3-e942-45a3-9736-c068ee4690ee'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:50.084926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e6fee821e7449b92c3df24de169fbc', 'request_id': 'req-6659cdd3-e942-45a3-9736-c068ee4690ee'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:50.096627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] The token cache is being invalidate because project 08e6fee821e7449b92c3df24de169fbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:32:50.096909 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:50.097538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6659cdd3-e942-45a3-9736-c068ee4690ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:50.104107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1140/2241] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:32:49 2022] DELETE /identity/v3/projects/08e6fee821e7449b92c3df24de169fbc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:32:53.218333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:53.219711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:53.232938 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f387fde-d1b7-4e86-b868-ae4abc6b9068 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:53.291364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:53.291993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:53.292270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:53.292575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] PATH_INFO: `/v3/users/24628c74de5b413cb493c328bf05866a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:53.293676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=24628c74de5b413cb493c328bf05866a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:53.295586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:53.309554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:32:53.342242 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24628c74de5b413cb493c328bf05866a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:53.349461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24628c74de5b413cb493c328bf05866a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:53.350104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24628c74de5b413cb493c328bf05866a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:53.358471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24628c74de5b413cb493c328bf05866a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:53.372736 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24628c74de5b413cb493c328bf05866a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:53.378701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1102/2242] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:32:53 2022] DELETE /identity/v3/users/24628c74de5b413cb493c328bf05866a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:32:53.383185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b552a8-e743-43f2-b641-06569076ca0f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:53.383659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b552a8-e743-43f2-b641-06569076ca0f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:53.383872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b552a8-e743-43f2-b641-06569076ca0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:53.403598 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-55b552a8-e743-43f2-b641-06569076ca0f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:32:53.404812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-55b552a8-e743-43f2-b641-06569076ca0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:53.461388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1141/2243] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:32:53.470911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:53.472472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:53.531925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a867500-8d21-4880-9bf3-51bd6da03fcd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:53.535816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41e953-1dc6-4191-83a2-d3c8bc098889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:53.536499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41e953-1dc6-4191-83a2-d3c8bc098889 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:53.536779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41e953-1dc6-4191-83a2-d3c8bc098889 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:53.537047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41e953-1dc6-4191-83a2-d3c8bc098889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:53.545750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41e953-1dc6-4191-83a2-d3c8bc098889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:53.547227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41e953-1dc6-4191-83a2-d3c8bc098889 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:53.557073 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1103/2244] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:32:53.833187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b552a8-e743-43f2-b641-06569076ca0f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:53.834674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-55b552a8-e743-43f2-b641-06569076ca0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:53.856780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b552a8-e743-43f2-b641-06569076ca0f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:53.871610 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:53.872328 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:53.872719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:53.873127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] PATH_INFO: `/v3/projects/c6a016a9695c47e18d335d633adf15a0` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:53.874330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a016a9695c47e18d335d633adf15a0)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:53.876357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:53.890556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:32:53.932689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a016a9695c47e18d335d633adf15a0', 'request_id': 'req-600bc5b3-dc3d-4d06-ab60-8552484680f3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:53.939856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a016a9695c47e18d335d633adf15a0', 'request_id': 'req-600bc5b3-dc3d-4d06-ab60-8552484680f3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:53.946272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a016a9695c47e18d335d633adf15a0', 'request_id': 'req-600bc5b3-dc3d-4d06-ab60-8552484680f3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:53.958346 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] The token cache is being invalidate because project c6a016a9695c47e18d335d633adf15a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:32:53.958642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:53.959225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-600bc5b3-dc3d-4d06-ab60-8552484680f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:32:53.960907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1142/2245] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:32:53 2022] DELETE /identity/v3/projects/c6a016a9695c47e18d335d633adf15a0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:32:53.965769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e3ccb-0a73-482a-bda7-fb42863f7510 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:53.966216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e3ccb-0a73-482a-bda7-fb42863f7510 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:53.966510 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e3ccb-0a73-482a-bda7-fb42863f7510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:53.986267 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7f7e3ccb-0a73-482a-bda7-fb42863f7510 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:32:53.987381 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7f7e3ccb-0a73-482a-bda7-fb42863f7510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:54.046313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1104/2246] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:32:54.054793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11bb93-0f1e-4a4e-9f47-c58ac0010c88 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:54.055069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11bb93-0f1e-4a4e-9f47-c58ac0010c88 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:54.055341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11bb93-0f1e-4a4e-9f47-c58ac0010c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:54.077706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ee11bb93-0f1e-4a4e-9f47-c58ac0010c88 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:32:54.078895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ee11bb93-0f1e-4a4e-9f47-c58ac0010c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:54.087173 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1143/2247] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:32:54.091404 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e3ccb-0a73-482a-bda7-fb42863f7510 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:54.101364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e3ccb-0a73-482a-bda7-fb42863f7510 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:54.105822 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341566b-380c-442a-a767-9feafdaa28c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:54.106403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341566b-380c-442a-a767-9feafdaa28c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:54.106676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341566b-380c-442a-a767-9feafdaa28c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:54.106943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341566b-380c-442a-a767-9feafdaa28c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:54.107579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341566b-380c-442a-a767-9feafdaa28c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:32:54.107872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341566b-380c-442a-a767-9feafdaa28c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:54.109482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341566b-380c-442a-a767-9feafdaa28c7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:54.120679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1105/2248] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:32:54.124701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11bb93-0f1e-4a4e-9f47-c58ac0010c88 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:54.133699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11bb93-0f1e-4a4e-9f47-c58ac0010c88 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:54.138410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfa5bb-2364-42dd-beb8-58b9f21db1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:54.138994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfa5bb-2364-42dd-beb8-58b9f21db1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:54.139271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfa5bb-2364-42dd-beb8-58b9f21db1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:54.139539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfa5bb-2364-42dd-beb8-58b9f21db1b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:54.140171 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfa5bb-2364-42dd-beb8-58b9f21db1b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:54.142193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfa5bb-2364-42dd-beb8-58b9f21db1b9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:54.172965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1144/2249] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:32:54 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:32:54.177259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341566b-380c-442a-a767-9feafdaa28c7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:54.187117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341566b-380c-442a-a767-9feafdaa28c7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:54.204972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:32:54.247941 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7f93c-71e4-4d35-9389-c721c8472b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:54.248619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7f93c-71e4-4d35-9389-c721c8472b52 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:54.248775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7f93c-71e4-4d35-9389-c721c8472b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:54.249048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7f93c-71e4-4d35-9389-c721c8472b52 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:54.249678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7f93c-71e4-4d35-9389-c721c8472b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:54.251588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7f93c-71e4-4d35-9389-c721c8472b52 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:54.272853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1106/2250] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:32:54 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:32:54.276872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfa5bb-2364-42dd-beb8-58b9f21db1b9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:54.287371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfa5bb-2364-42dd-beb8-58b9f21db1b9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:54.291877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e00165-2f45-4b15-a9d9-1605858d5af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:54.292501 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e00165-2f45-4b15-a9d9-1605858d5af8 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:54.292777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e00165-2f45-4b15-a9d9-1605858d5af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:54.293043 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e00165-2f45-4b15-a9d9-1605858d5af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:54.293959 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e00165-2f45-4b15-a9d9-1605858d5af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:54.295874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e00165-2f45-4b15-a9d9-1605858d5af8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:54.306059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1145/2251] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:32:54.309856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7f93c-71e4-4d35-9389-c721c8472b52 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:54.320149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7f93c-71e4-4d35-9389-c721c8472b52 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:54.325255 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b65eb-172d-4366-a098-3069212a622a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:54.326085 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b65eb-172d-4366-a098-3069212a622a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:54.326432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b65eb-172d-4366-a098-3069212a622a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:54.326701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b65eb-172d-4366-a098-3069212a622a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:54.327356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b65eb-172d-4366-a098-3069212a622a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:54.329828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b65eb-172d-4366-a098-3069212a622a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:54.341712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1107/2252] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:32:54.345690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e00165-2f45-4b15-a9d9-1605858d5af8 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:54.355729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e00165-2f45-4b15-a9d9-1605858d5af8 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:54.360547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc927f98-0c3c-441d-8368-3a07ccd1dbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:54.361197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc927f98-0c3c-441d-8368-3a07ccd1dbc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:54.361471 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc927f98-0c3c-441d-8368-3a07ccd1dbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:54.361735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc927f98-0c3c-441d-8368-3a07ccd1dbc4 admin admin] PATH_INFO: `/v3/projects/4ff43b0481564647b7b24b5ba1f7b10b/users/3e09915f31ab48dab415e5fa932c7001/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:54.376407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc927f98-0c3c-441d-8368-3a07ccd1dbc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff43b0481564647b7b24b5ba1f7b10b, user_id=3e09915f31ab48dab415e5fa932c7001, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:54.379008 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc927f98-0c3c-441d-8368-3a07ccd1dbc4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:54.393996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1146/2253] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:32:54 2022] PUT /identity/v3/projects/4ff43b0481564647b7b24b5ba1f7b10b/users/3e09915f31ab48dab415e5fa932c7001/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:32:54.405585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08964a-a45d-480f-bbfc-447d227667e7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:54.405870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08964a-a45d-480f-bbfc-447d227667e7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:54.406138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08964a-a45d-480f-bbfc-447d227667e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:54.437250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3f08964a-a45d-480f-bbfc-447d227667e7 None None] MFA Rules not processed for user `3e09915f31ab48dab415e5fa932c7001`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:32:54.438435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3f08964a-a45d-480f-bbfc-447d227667e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:54.502774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1108/2254] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:32:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:32:54.513200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc927f98-0c3c-441d-8368-3a07ccd1dbc4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:32:54.514862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dc927f98-0c3c-441d-8368-3a07ccd1dbc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:32:54.571055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc927f98-0c3c-441d-8368-3a07ccd1dbc4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:32:54.575059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f4c8c-8dc0-404e-ba99-9c2215198b72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:32:54.575666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f4c8c-8dc0-404e-ba99-9c2215198b72 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:32:54.575951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f4c8c-8dc0-404e-ba99-9c2215198b72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:32:54.576259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f4c8c-8dc0-404e-ba99-9c2215198b72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:32:54.586111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f4c8c-8dc0-404e-ba99-9c2215198b72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:32:54.587598 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f4c8c-8dc0-404e-ba99-9c2215198b72 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:32:54.597013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1147/2255] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:32:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:33:04.499652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:33:04.592750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:33:07.905403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08964a-a45d-480f-bbfc-447d227667e7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:33:07.967080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08964a-a45d-480f-bbfc-447d227667e7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:33:07.971455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:33:07.972071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:33:07.972345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:33:07.972718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] PATH_INFO: `/v3/users/3e09915f31ab48dab415e5fa932c7001` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:33:07.974331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e09915f31ab48dab415e5fa932c7001)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:33:07.976228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:33:08.021731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e09915f31ab48dab415e5fa932c7001'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:08.028637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e09915f31ab48dab415e5fa932c7001'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:08.029291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e09915f31ab48dab415e5fa932c7001'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:08.036965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e09915f31ab48dab415e5fa932c7001'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:08.049301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e09915f31ab48dab415e5fa932c7001'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:08.055709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1109/2256] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:33:07 2022] DELETE /identity/v3/users/3e09915f31ab48dab415e5fa932c7001 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:33:08.060360 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b787ed-3596-422a-ad4f-955e452670cd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:33:08.060870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b787ed-3596-422a-ad4f-955e452670cd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:33:08.061002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b787ed-3596-422a-ad4f-955e452670cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:33:08.079205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-91b787ed-3596-422a-ad4f-955e452670cd None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:33:08.080415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-91b787ed-3596-422a-ad4f-955e452670cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:33:08.136035 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1148/2257] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:33:08.145628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:33:08.147007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:33:08.203844 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28525df-45d2-4dad-8eb7-63449ee8948e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:33:08.207462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de35d7-23fe-485b-9f20-d553acea4476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:33:08.208054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de35d7-23fe-485b-9f20-d553acea4476 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:33:08.208354 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de35d7-23fe-485b-9f20-d553acea4476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:33:08.208658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de35d7-23fe-485b-9f20-d553acea4476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:33:08.217402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de35d7-23fe-485b-9f20-d553acea4476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:33:08.218832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de35d7-23fe-485b-9f20-d553acea4476 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:33:08.228606 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1110/2258] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:33:08.489077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b787ed-3596-422a-ad4f-955e452670cd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:33:08.490518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-91b787ed-3596-422a-ad4f-955e452670cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:33:08.502864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b787ed-3596-422a-ad4f-955e452670cd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:33:08.507448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:33:08.508102 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:33:08.508396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:33:08.508675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] PATH_INFO: `/v3/projects/4ff43b0481564647b7b24b5ba1f7b10b` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:33:08.509800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff43b0481564647b7b24b5ba1f7b10b)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:33:08.511812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:33:08.557365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff43b0481564647b7b24b5ba1f7b10b', 'request_id': 'req-f31bfed3-fead-4113-a5f2-b7c197043524'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:08.563109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff43b0481564647b7b24b5ba1f7b10b', 'request_id': 'req-f31bfed3-fead-4113-a5f2-b7c197043524'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:08.569185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff43b0481564647b7b24b5ba1f7b10b', 'request_id': 'req-f31bfed3-fead-4113-a5f2-b7c197043524'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:08.580996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] The token cache is being invalidate because project 4ff43b0481564647b7b24b5ba1f7b10b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:33:08.581282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:08.581892 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f31bfed3-fead-4113-a5f2-b7c197043524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:08.583858 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1149/2259] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:33:08 2022] DELETE /identity/v3/projects/4ff43b0481564647b7b24b5ba1f7b10b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:33:17.907315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de35d7-23fe-485b-9f20-d553acea4476 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:33:17.908742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-67de35d7-23fe-485b-9f20-d553acea4476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:33:17.921164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de35d7-23fe-485b-9f20-d553acea4476 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:33:17.939357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:33:17.977835 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:33:17.978446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:33:17.978719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:33:17.978985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] PATH_INFO: `/v3/users/ab8c2d9d39574d8ca94114b7eee59f16` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:33:17.980071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab8c2d9d39574d8ca94114b7eee59f16)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:33:17.982007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:33:18.026321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab8c2d9d39574d8ca94114b7eee59f16'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:18.032884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab8c2d9d39574d8ca94114b7eee59f16'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:18.033514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab8c2d9d39574d8ca94114b7eee59f16'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:18.041204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab8c2d9d39574d8ca94114b7eee59f16'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:18.053535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab8c2d9d39574d8ca94114b7eee59f16'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:18.059754 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1111/2260] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:33:17 2022] DELETE /identity/v3/users/ab8c2d9d39574d8ca94114b7eee59f16 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:33:18.064289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cbc83-344c-4c30-b130-636774a4341a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:33:18.064792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cbc83-344c-4c30-b130-636774a4341a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:33:18.064896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cbc83-344c-4c30-b130-636774a4341a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:33:18.083429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-e14cbc83-344c-4c30-b130-636774a4341a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:33:18.084669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e14cbc83-344c-4c30-b130-636774a4341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:33:18.139890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1150/2261] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:33:18.149316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:33:18.150706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:33:18.206747 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8646800-c8dc-42b5-ab53-e43d7e21b7c0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:33:18.210321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79db962-f955-4764-9f95-57a928a018fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:33:18.210917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79db962-f955-4764-9f95-57a928a018fa service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:33:18.211192 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79db962-f955-4764-9f95-57a928a018fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:33:18.211463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79db962-f955-4764-9f95-57a928a018fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:33:18.220241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79db962-f955-4764-9f95-57a928a018fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:33:18.221678 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79db962-f955-4764-9f95-57a928a018fa service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:33:18.231335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1112/2262] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:33:18.497102 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cbc83-344c-4c30-b130-636774a4341a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:33:18.498562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e14cbc83-344c-4c30-b130-636774a4341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:33:18.512548 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cbc83-344c-4c30-b130-636774a4341a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:33:18.517452 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:33:18.518158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:33:18.518503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:33:18.518778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] PATH_INFO: `/v3/projects/3c3d05188b0747a2920a3e9101d5828d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:33:18.519978 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c3d05188b0747a2920a3e9101d5828d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:33:18.522394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:33:18.543923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:33:18.569847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c3d05188b0747a2920a3e9101d5828d', 'request_id': 'req-dc28feea-b642-4c23-8980-0efb6c47edaf'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:18.575259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c3d05188b0747a2920a3e9101d5828d', 'request_id': 'req-dc28feea-b642-4c23-8980-0efb6c47edaf'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:18.581274 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c3d05188b0747a2920a3e9101d5828d', 'request_id': 'req-dc28feea-b642-4c23-8980-0efb6c47edaf'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:18.593865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] The token cache is being invalidate because project 3c3d05188b0747a2920a3e9101d5828d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:33:18.594150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:18.594753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:33:18.596203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1151/2263] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:33:18 2022] DELETE /identity/v3/projects/3c3d05188b0747a2920a3e9101d5828d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:33:28.227256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:33:28.590862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:33:33.061427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ef020-db35-4966-a781-fcebbc3d5bf8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:33:33.061887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ef020-db35-4966-a781-fcebbc3d5bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:33:33.062005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ef020-db35-4966-a781-fcebbc3d5bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:33:33.081225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8e3ef020-db35-4966-a781-fcebbc3d5bf8 None None] MFA Rules not processed for user `ccbdadc310fc4bf8a6679bcb39b34ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:33:33.082419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8e3ef020-db35-4966-a781-fcebbc3d5bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:33:33.134525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1113/2264] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:33:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:33:43.127595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=25 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:33:59.993692 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:33:59.995105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:00.055764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28feea-b642-4c23-8980-0efb6c47edaf admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:00.059293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f16cbb-8234-499b-98d3-5273cdb80855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:00.060187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f16cbb-8234-499b-98d3-5273cdb80855 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:00.061842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f16cbb-8234-499b-98d3-5273cdb80855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:00.061842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f16cbb-8234-499b-98d3-5273cdb80855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:00.062503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-21f16cbb-8234-499b-98d3-5273cdb80855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:00.136094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f16cbb-8234-499b-98d3-5273cdb80855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:00.137551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f16cbb-8234-499b-98d3-5273cdb80855 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:00.149126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1152/2265] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:33:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:34:01.350961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ef020-db35-4966-a781-fcebbc3d5bf8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:01.352646 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8e3ef020-db35-4966-a781-fcebbc3d5bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:01.364426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ef020-db35-4966-a781-fcebbc3d5bf8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:01.368689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da85427-1b7c-485f-b70c-aa35bf38ccb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:01.369300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da85427-1b7c-485f-b70c-aa35bf38ccb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:01.369579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da85427-1b7c-485f-b70c-aa35bf38ccb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:01.369882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da85427-1b7c-485f-b70c-aa35bf38ccb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:01.371618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5da85427-1b7c-485f-b70c-aa35bf38ccb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:01.385240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da85427-1b7c-485f-b70c-aa35bf38ccb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:01.386690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da85427-1b7c-485f-b70c-aa35bf38ccb6 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:01.398197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1114/2266] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:34:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:34:06.836416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60c72f-193b-463e-b7fb-8ac55073728e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:06.836932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60c72f-193b-463e-b7fb-8ac55073728e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:06.837041 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60c72f-193b-463e-b7fb-8ac55073728e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:06.855775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ea60c72f-193b-463e-b7fb-8ac55073728e None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:34:06.857069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ea60c72f-193b-463e-b7fb-8ac55073728e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:06.912041 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1153/2267] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:34:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:34:06.958907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da85427-1b7c-485f-b70c-aa35bf38ccb6 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:06.960324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5da85427-1b7c-485f-b70c-aa35bf38ccb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:07.028314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da85427-1b7c-485f-b70c-aa35bf38ccb6 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:07.033173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8c11e-b997-461c-88d5-7d0e6e38785e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:07.033767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c11e-b997-461c-88d5-7d0e6e38785e service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:07.034040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c11e-b997-461c-88d5-7d0e6e38785e service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:07.034693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c11e-b997-461c-88d5-7d0e6e38785e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:07.042593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8c11e-b997-461c-88d5-7d0e6e38785e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:07.044156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8c11e-b997-461c-88d5-7d0e6e38785e service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:07.051894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1115/2268] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:34:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:34:16.905206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=79 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:34:17.048774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=38 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:34:18.117227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60c72f-193b-463e-b7fb-8ac55073728e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:18.119244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ea60c72f-193b-463e-b7fb-8ac55073728e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:18.134552 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60c72f-193b-463e-b7fb-8ac55073728e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:18.139909 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565edba2-1b70-46c3-81a6-0fb5827e9506 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:18.140568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565edba2-1b70-46c3-81a6-0fb5827e9506 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:18.140852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565edba2-1b70-46c3-81a6-0fb5827e9506 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:18.141124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565edba2-1b70-46c3-81a6-0fb5827e9506 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:18.142791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-565edba2-1b70-46c3-81a6-0fb5827e9506 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:18.200034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565edba2-1b70-46c3-81a6-0fb5827e9506 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:18.201661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565edba2-1b70-46c3-81a6-0fb5827e9506 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:18.211115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1154/2269] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:34:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:34:21.194804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8c11e-b997-461c-88d5-7d0e6e38785e service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:21.203703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8c11e-b997-461c-88d5-7d0e6e38785e service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:21.207179 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52defb9-ed7b-4bee-8ea9-2d045ba00155 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:21.207755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52defb9-ed7b-4bee-8ea9-2d045ba00155 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:21.208026 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52defb9-ed7b-4bee-8ea9-2d045ba00155 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:21.208288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52defb9-ed7b-4bee-8ea9-2d045ba00155 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:21.210555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b52defb9-ed7b-4bee-8ea9-2d045ba00155 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:21.219707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52defb9-ed7b-4bee-8ea9-2d045ba00155 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:21.221204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52defb9-ed7b-4bee-8ea9-2d045ba00155 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:21.229719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1116/2270] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:34:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:34:21.354139 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9c51d-90f4-4c0b-b578-12017e3cf089 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:21.354698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9c51d-90f4-4c0b-b578-12017e3cf089 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:21.354785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9c51d-90f4-4c0b-b578-12017e3cf089 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:21.356087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1155/2271] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:34:21.362840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b02c81-a8b8-4572-99f8-c39af16a053b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:21.363317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b02c81-a8b8-4572-99f8-c39af16a053b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:21.363653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b02c81-a8b8-4572-99f8-c39af16a053b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:21.383207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-97b02c81-a8b8-4572-99f8-c39af16a053b None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:34:21.384661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-97b02c81-a8b8-4572-99f8-c39af16a053b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:21.393860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1117/2272] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:34:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:34:21.408061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9c51d-90f4-4c0b-b578-12017e3cf089 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:21.410071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-46e9c51d-90f4-4c0b-b578-12017e3cf089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:21.473025 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9c51d-90f4-4c0b-b578-12017e3cf089 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:21.476623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb1e06-9b1e-475c-ac5d-b2b89a33e6d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a9b371434243daa4176215daefb79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:21.477232 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb1e06-9b1e-475c-ac5d-b2b89a33e6d9 service glance] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:21.477509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb1e06-9b1e-475c-ac5d-b2b89a33e6d9 service glance] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:21.477780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb1e06-9b1e-475c-ac5d-b2b89a33e6d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:21.488002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb1e06-9b1e-475c-ac5d-b2b89a33e6d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:21.489465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb1e06-9b1e-475c-ac5d-b2b89a33e6d9 service glance] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:21.500778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1156/2273] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:34:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:34:21.569048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a8312-d508-40d1-ac57-f9bd3189982e None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:21.569331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a8312-d508-40d1-ac57-f9bd3189982e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:21.569639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a8312-d508-40d1-ac57-f9bd3189982e None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:21.570983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1118/2274] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:34:21.575744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a1758-0225-45c7-8ac0-09fb7ec4ac1e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:21.576055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a1758-0225-45c7-8ac0-09fb7ec4ac1e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:21.576351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a1758-0225-45c7-8ac0-09fb7ec4ac1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:21.596956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c45a1758-0225-45c7-8ac0-09fb7ec4ac1e None None] MFA Rules not processed for user `18ea49d1f6c040f18511535b7e51a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:34:21.598164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c45a1758-0225-45c7-8ac0-09fb7ec4ac1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:21.604887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1157/2275] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:34:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:34:21.618046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a8312-d508-40d1-ac57-f9bd3189982e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:21.628879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a8312-d508-40d1-ac57-f9bd3189982e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:21.632845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886148c2-9c29-4a09-a77f-bc939de11610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:21.633433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886148c2-9c29-4a09-a77f-bc939de11610 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:21.633711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886148c2-9c29-4a09-a77f-bc939de11610 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:21.633980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886148c2-9c29-4a09-a77f-bc939de11610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:21.640838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886148c2-9c29-4a09-a77f-bc939de11610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:21.642249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886148c2-9c29-4a09-a77f-bc939de11610 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:21.650341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1119/2276] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:34:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:34:21.735957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f12d2-8db2-4521-9c67-2b0f3cd1be43 None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:21.736262 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f12d2-8db2-4521-9c67-2b0f3cd1be43 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:21.736576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f12d2-8db2-4521-9c67-2b0f3cd1be43 None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:21.737918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1158/2277] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:34:21.742797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3fad8-a3a6-4092-a7b8-a43f559fa085 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:21.743080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3fad8-a3a6-4092-a7b8-a43f559fa085 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:21.743349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3fad8-a3a6-4092-a7b8-a43f559fa085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:21.763089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-eed3fad8-a3a6-4092-a7b8-a43f559fa085 None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:34:21.764279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eed3fad8-a3a6-4092-a7b8-a43f559fa085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:21.773978 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1120/2278] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:34:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:34:21.784575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f12d2-8db2-4521-9c67-2b0f3cd1be43 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:21.795238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f12d2-8db2-4521-9c67-2b0f3cd1be43 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:21.799185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dda5f9-6903-41ab-903f-964c8b040760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:21.799799 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dda5f9-6903-41ab-903f-964c8b040760 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:21.800075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dda5f9-6903-41ab-903f-964c8b040760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:21.800341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dda5f9-6903-41ab-903f-964c8b040760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:21.810872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dda5f9-6903-41ab-903f-964c8b040760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:21.812268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dda5f9-6903-41ab-903f-964c8b040760 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:21.823362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1159/2279] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:34:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:34:21.884504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3fad8-a3a6-4092-a7b8-a43f559fa085 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:21.885874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eed3fad8-a3a6-4092-a7b8-a43f559fa085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:21.951449 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3fad8-a3a6-4092-a7b8-a43f559fa085 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:21.956461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85586c-2b8d-4768-9f1d-1f3be13c07ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:21.957053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85586c-2b8d-4768-9f1d-1f3be13c07ae service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:21.957332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85586c-2b8d-4768-9f1d-1f3be13c07ae service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:21.957603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85586c-2b8d-4768-9f1d-1f3be13c07ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:21.959314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ce85586c-2b8d-4768-9f1d-1f3be13c07ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:21.970202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85586c-2b8d-4768-9f1d-1f3be13c07ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:21.971756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85586c-2b8d-4768-9f1d-1f3be13c07ae service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:21.981407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1121/2280] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:34:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:34:24.791418 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca08fe-4208-4ebb-b238-9e38f16c53a9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:24.791888 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca08fe-4208-4ebb-b238-9e38f16c53a9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:24.792006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca08fe-4208-4ebb-b238-9e38f16c53a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:24.811255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-baca08fe-4208-4ebb-b238-9e38f16c53a9 None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:34:24.812361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-baca08fe-4208-4ebb-b238-9e38f16c53a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:24.820005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1160/2281] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:34:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:34:24.831601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85586c-2b8d-4768-9f1d-1f3be13c07ae service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:24.842616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85586c-2b8d-4768-9f1d-1f3be13c07ae service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:24.847406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60f91c-a975-465e-93dd-1b87768ab96d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:24.847984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60f91c-a975-465e-93dd-1b87768ab96d service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:24.848256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60f91c-a975-465e-93dd-1b87768ab96d service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:24.848554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60f91c-a975-465e-93dd-1b87768ab96d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:24.856450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60f91c-a975-465e-93dd-1b87768ab96d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:24.858016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60f91c-a975-465e-93dd-1b87768ab96d service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:24.866612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1122/2282] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:34:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:34:30.263916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca08fe-4208-4ebb-b238-9e38f16c53a9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:30.277009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca08fe-4208-4ebb-b238-9e38f16c53a9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:30.283296 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd53eee-78c7-4d72-bcdb-ea060ed193bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:30.284036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd53eee-78c7-4d72-bcdb-ea060ed193bc service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:30.284612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd53eee-78c7-4d72-bcdb-ea060ed193bc service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:30.284883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd53eee-78c7-4d72-bcdb-ea060ed193bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:30.286689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bcd53eee-78c7-4d72-bcdb-ea060ed193bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:30.349146 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd53eee-78c7-4d72-bcdb-ea060ed193bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:30.350770 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd53eee-78c7-4d72-bcdb-ea060ed193bc service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:30.360452 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1161/2283] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:34:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:34:34.862729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=56 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:34:40.355826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=94 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:34:51.587295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60f91c-a975-465e-93dd-1b87768ab96d service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:51.596623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60f91c-a975-465e-93dd-1b87768ab96d service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:51.600171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73657a25-cd8c-4627-b823-30ac496c0f0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:51.601018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73657a25-cd8c-4627-b823-30ac496c0f0a service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:51.601445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73657a25-cd8c-4627-b823-30ac496c0f0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:51.601728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73657a25-cd8c-4627-b823-30ac496c0f0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:51.603419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-73657a25-cd8c-4627-b823-30ac496c0f0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:51.612566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73657a25-cd8c-4627-b823-30ac496c0f0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:51.613980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73657a25-cd8c-4627-b823-30ac496c0f0a service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:51.621940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1123/2284] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:34:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:34:54.182648 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d632f10-36c8-449e-96f4-f796022ec69f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:54.183117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d632f10-36c8-449e-96f4-f796022ec69f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:54.183229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d632f10-36c8-449e-96f4-f796022ec69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:54.202074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1d632f10-36c8-449e-96f4-f796022ec69f None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:34:54.203257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1d632f10-36c8-449e-96f4-f796022ec69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:34:54.210781 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1162/2285] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 30 19:34:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:34:54.224040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73657a25-cd8c-4627-b823-30ac496c0f0a service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:34:54.236521 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73657a25-cd8c-4627-b823-30ac496c0f0a service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:34:54.241516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0517758-f827-4133-aef2-2c27357081d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:34:54.242089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0517758-f827-4133-aef2-2c27357081d1 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:34:54.242363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0517758-f827-4133-aef2-2c27357081d1 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:34:54.242682 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0517758-f827-4133-aef2-2c27357081d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:34:54.250633 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0517758-f827-4133-aef2-2c27357081d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:34:54.252188 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0517758-f827-4133-aef2-2c27357081d1 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:34:54.260129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1124/2286] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:34:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:35:04.196638 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:35:04.256935 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=12 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:35:09.870895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1525c-f125-41a4-8288-b7f4d446d9e5 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:09.871439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1525c-f125-41a4-8288-b7f4d446d9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:09.871439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1525c-f125-41a4-8288-b7f4d446d9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:09.890925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5bd1525c-f125-41a4-8288-b7f4d446d9e5 None None] MFA Rules not processed for user `624738daa9db4aff885d71ea2d65fb26`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:35:09.892104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5bd1525c-f125-41a4-8288-b7f4d446d9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:09.899286 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1163/2287] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:35:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:35:16.211922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0517758-f827-4133-aef2-2c27357081d1 service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:35:16.226301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0517758-f827-4133-aef2-2c27357081d1 service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:35:16.232237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16960486-308a-4840-b360-03f2efee1267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:35:16.232973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16960486-308a-4840-b360-03f2efee1267 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:16.233387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16960486-308a-4840-b360-03f2efee1267 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:16.233763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16960486-308a-4840-b360-03f2efee1267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:16.235602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-16960486-308a-4840-b360-03f2efee1267 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:16.248641 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16960486-308a-4840-b360-03f2efee1267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:35:16.250347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16960486-308a-4840-b360-03f2efee1267 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:35:16.261018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1125/2288] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:35:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:35:19.885443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:35:22.458603 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1525c-f125-41a4-8288-b7f4d446d9e5 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:35:22.460134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5bd1525c-f125-41a4-8288-b7f4d446d9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:22.475700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1525c-f125-41a4-8288-b7f4d446d9e5 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:35:22.481148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcf682-9146-4f9e-85f0-49c9a2a9aabf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:35:22.481751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcf682-9146-4f9e-85f0-49c9a2a9aabf service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:22.482029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcf682-9146-4f9e-85f0-49c9a2a9aabf service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:22.482302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcf682-9146-4f9e-85f0-49c9a2a9aabf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:22.484005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8abcf682-9146-4f9e-85f0-49c9a2a9aabf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:22.543857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abcf682-9146-4f9e-85f0-49c9a2a9aabf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:35:22.545537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abcf682-9146-4f9e-85f0-49c9a2a9aabf service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:35:22.555278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1164/2289] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:35:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:35:23.026415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99593b1c-b91c-452f-8c02-034c6906c211 None None] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:23.026971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99593b1c-b91c-452f-8c02-034c6906c211 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:23.027117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99593b1c-b91c-452f-8c02-034c6906c211 None None] PATH_INFO: `/` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:23.028592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1126/2290] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:35:23.034924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb2d31-60c5-4d01-b160-f348a07b9343 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:23.035276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb2d31-60c5-4d01-b160-f348a07b9343 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:23.036674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb2d31-60c5-4d01-b160-f348a07b9343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:23.056499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-60fb2d31-60c5-4d01-b160-f348a07b9343 None None] MFA Rules not processed for user `18ea49d1f6c040f18511535b7e51a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:35:23.057704 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-60fb2d31-60c5-4d01-b160-f348a07b9343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:23.064141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1165/2291] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:35:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:35:23.076899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99593b1c-b91c-452f-8c02-034c6906c211 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:35:23.088105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99593b1c-b91c-452f-8c02-034c6906c211 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:35:23.092946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e68334-716d-46e7-94c8-3e8fcea4e889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:35:23.093557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e68334-716d-46e7-94c8-3e8fcea4e889 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:23.093828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e68334-716d-46e7-94c8-3e8fcea4e889 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:23.094093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e68334-716d-46e7-94c8-3e8fcea4e889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:23.101593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e68334-716d-46e7-94c8-3e8fcea4e889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:35:23.103043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e68334-716d-46e7-94c8-3e8fcea4e889 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:35:23.111046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1127/2292] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:35:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:35:23.192402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b5a52-03ce-4734-a7f3-01d02ac4ad4e None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:23.192689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b5a52-03ce-4734-a7f3-01d02ac4ad4e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:23.192964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b5a52-03ce-4734-a7f3-01d02ac4ad4e None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:23.194374 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1166/2293] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:35:23.200339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cbd5e-d29d-449d-8318-3142c7330817 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:23.200652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cbd5e-d29d-449d-8318-3142c7330817 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:23.200922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cbd5e-d29d-449d-8318-3142c7330817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:23.219865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bd1cbd5e-d29d-449d-8318-3142c7330817 None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:35:23.221063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bd1cbd5e-d29d-449d-8318-3142c7330817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:23.230139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1128/2294] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:35:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:35:23.240278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b5a52-03ce-4734-a7f3-01d02ac4ad4e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:35:23.251616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b5a52-03ce-4734-a7f3-01d02ac4ad4e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:35:23.255221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52699ec-bafa-460d-bfd8-60986d514394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:35:23.255805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52699ec-bafa-460d-bfd8-60986d514394 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:23.256080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52699ec-bafa-460d-bfd8-60986d514394 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:23.256345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52699ec-bafa-460d-bfd8-60986d514394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:23.266777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52699ec-bafa-460d-bfd8-60986d514394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:35:23.268186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52699ec-bafa-460d-bfd8-60986d514394 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:35:23.278503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1167/2295] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:35:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:35:24.949472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cbd5e-d29d-449d-8318-3142c7330817 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:35:24.964197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cbd5e-d29d-449d-8318-3142c7330817 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:35:24.970147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f1338-b92f-41af-a5d1-9cb600cc6b1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:35:24.970812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f1338-b92f-41af-a5d1-9cb600cc6b1b service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:24.971139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f1338-b92f-41af-a5d1-9cb600cc6b1b service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:24.971561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f1338-b92f-41af-a5d1-9cb600cc6b1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:24.973566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b75f1338-b92f-41af-a5d1-9cb600cc6b1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:24.985867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f1338-b92f-41af-a5d1-9cb600cc6b1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:35:24.987960 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f1338-b92f-41af-a5d1-9cb600cc6b1b service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:35:25.000108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1129/2296] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:35:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:35:33.274036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=37 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:35:34.995055 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=21 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:35:38.868833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52699ec-bafa-460d-bfd8-60986d514394 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:35:38.879876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52699ec-bafa-460d-bfd8-60986d514394 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:35:38.885432 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02d6a1-92ea-4a20-8529-7e24d5a9f77b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:35:38.886023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02d6a1-92ea-4a20-8529-7e24d5a9f77b service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:38.886300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02d6a1-92ea-4a20-8529-7e24d5a9f77b service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:38.886570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02d6a1-92ea-4a20-8529-7e24d5a9f77b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:38.888271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0a02d6a1-92ea-4a20-8529-7e24d5a9f77b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:38.898785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02d6a1-92ea-4a20-8529-7e24d5a9f77b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:35:38.900351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02d6a1-92ea-4a20-8529-7e24d5a9f77b service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:35:38.909246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1168/2297] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:35:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:35:40.589997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f1338-b92f-41af-a5d1-9cb600cc6b1b service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:35:40.600757 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f1338-b92f-41af-a5d1-9cb600cc6b1b service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:35:40.606094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb988fe-d622-4b1d-96ae-aeda855dbf73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:35:40.606672 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb988fe-d622-4b1d-96ae-aeda855dbf73 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:40.606969 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb988fe-d622-4b1d-96ae-aeda855dbf73 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:40.607236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb988fe-d622-4b1d-96ae-aeda855dbf73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:40.609021 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7bb988fe-d622-4b1d-96ae-aeda855dbf73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:40.665671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb988fe-d622-4b1d-96ae-aeda855dbf73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:35:40.667305 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb988fe-d622-4b1d-96ae-aeda855dbf73 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:35:40.676187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1130/2298] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:35:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:35:43.185916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02d6a1-92ea-4a20-8529-7e24d5a9f77b service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:35:43.197783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02d6a1-92ea-4a20-8529-7e24d5a9f77b service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:35:43.203706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c906a-f881-4ebe-aa87-83575c3388b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:35:43.204577 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c906a-f881-4ebe-aa87-83575c3388b0 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:43.205065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c906a-f881-4ebe-aa87-83575c3388b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:43.205502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c906a-f881-4ebe-aa87-83575c3388b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:43.207542 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c95c906a-f881-4ebe-aa87-83575c3388b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:43.221790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c906a-f881-4ebe-aa87-83575c3388b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:35:43.223597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c906a-f881-4ebe-aa87-83575c3388b0 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:35:43.236088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1169/2299] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:35:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:35:50.671993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=28 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:35:52.510218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787559f0-47e8-46ef-bf59-b89de06adf7e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:52.510840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787559f0-47e8-46ef-bf59-b89de06adf7e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:52.510840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787559f0-47e8-46ef-bf59-b89de06adf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:52.529223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-787559f0-47e8-46ef-bf59-b89de06adf7e None None] MFA Rules not processed for user `ccbdadc310fc4bf8a6679bcb39b34ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:35:52.530388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-787559f0-47e8-46ef-bf59-b89de06adf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:52.537268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1131/2300] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:35:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:35:52.838130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c906a-f881-4ebe-aa87-83575c3388b0 service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:35:52.847304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c906a-f881-4ebe-aa87-83575c3388b0 service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:35:52.850923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f74750-4162-4ff2-b91f-1f8981335e6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:35:52.851509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f74750-4162-4ff2-b91f-1f8981335e6b service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:35:52.851786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f74750-4162-4ff2-b91f-1f8981335e6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:35:52.852055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f74750-4162-4ff2-b91f-1f8981335e6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:35:52.853774 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-04f74750-4162-4ff2-b91f-1f8981335e6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:35:52.862936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f74750-4162-4ff2-b91f-1f8981335e6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:35:52.864335 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f74750-4162-4ff2-b91f-1f8981335e6b service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:35:52.872571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1170/2301] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:35:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:36:02.523999 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:36:02.869157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=18 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:36:21.791562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ffce4-5b16-44f5-ac44-676015bc777c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:21.792080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ffce4-5b16-44f5-ac44-676015bc777c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:21.792352 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ffce4-5b16-44f5-ac44-676015bc777c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:21.813301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9b5ffce4-5b16-44f5-ac44-676015bc777c None None] MFA Rules not processed for user `f7aa40f3966d4c97ab6a1a764722969d`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:36:21.814472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b5ffce4-5b16-44f5-ac44-676015bc777c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:36:21.821109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1132/2302] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:36:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:36:31.808639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:36:53.299440 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f74750-4162-4ff2-b91f-1f8981335e6b service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:53.307821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f74750-4162-4ff2-b91f-1f8981335e6b service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:53.311354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803f6d3-72e1-46ba-a0e2-1bb5dfc68a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:53.311938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803f6d3-72e1-46ba-a0e2-1bb5dfc68a46 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:53.312208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803f6d3-72e1-46ba-a0e2-1bb5dfc68a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:53.312509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803f6d3-72e1-46ba-a0e2-1bb5dfc68a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:53.314158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5803f6d3-72e1-46ba-a0e2-1bb5dfc68a46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:36:53.373771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803f6d3-72e1-46ba-a0e2-1bb5dfc68a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:53.375176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803f6d3-72e1-46ba-a0e2-1bb5dfc68a46 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:53.385039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1171/2303] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:36:56.508063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ffce4-5b16-44f5-ac44-676015bc777c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:56.522103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ffce4-5b16-44f5-ac44-676015bc777c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:56.527727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1352c-3f6a-41fd-963f-9f1b983d7b1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:56.528578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1352c-3f6a-41fd-963f-9f1b983d7b1c service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:56.528923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1352c-3f6a-41fd-963f-9f1b983d7b1c service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:56.529973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1352c-3f6a-41fd-963f-9f1b983d7b1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:56.531784 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5bd1352c-3f6a-41fd-963f-9f1b983d7b1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:36:56.545125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1352c-3f6a-41fd-963f-9f1b983d7b1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:56.546985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1352c-3f6a-41fd-963f-9f1b983d7b1c service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:56.558598 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1133/2304] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:36:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:36:56.579173 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803f6d3-72e1-46ba-a0e2-1bb5dfc68a46 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:56.580621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5803f6d3-72e1-46ba-a0e2-1bb5dfc68a46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:36:56.592872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803f6d3-72e1-46ba-a0e2-1bb5dfc68a46 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:56.598215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38922d-4847-4af7-bc13-3403094372c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:56.598215 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38922d-4847-4af7-bc13-3403094372c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:56.598497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38922d-4847-4af7-bc13-3403094372c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:56.598953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38922d-4847-4af7-bc13-3403094372c6 admin admin] PATH_INFO: `/v3/users/624738daa9db4aff885d71ea2d65fb26` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:56.600620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38922d-4847-4af7-bc13-3403094372c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=624738daa9db4aff885d71ea2d65fb26)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:56.602519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38922d-4847-4af7-bc13-3403094372c6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:56.649495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9e38922d-4847-4af7-bc13-3403094372c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624738daa9db4aff885d71ea2d65fb26'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:36:56.656233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9e38922d-4847-4af7-bc13-3403094372c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624738daa9db4aff885d71ea2d65fb26'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:36:56.656915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9e38922d-4847-4af7-bc13-3403094372c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624738daa9db4aff885d71ea2d65fb26'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:36:56.665790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9e38922d-4847-4af7-bc13-3403094372c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624738daa9db4aff885d71ea2d65fb26'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:36:56.678516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-9e38922d-4847-4af7-bc13-3403094372c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624738daa9db4aff885d71ea2d65fb26'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:36:56.684338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1172/2305] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:36:56 2022] DELETE /identity/v3/users/624738daa9db4aff885d71ea2d65fb26 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:36:56.853948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1352c-3f6a-41fd-963f-9f1b983d7b1c service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:56.865256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1352c-3f6a-41fd-963f-9f1b983d7b1c service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:56.924937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:56.925613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:56.925932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:56.926236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] PATH_INFO: `/v3/projects/de44d50bb79d48189cd74cad8079e01f` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:56.927443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de44d50bb79d48189cd74cad8079e01f)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:56.929642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:56.980758 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de44d50bb79d48189cd74cad8079e01f', 'request_id': 'req-158fa8ef-d701-4e67-95fc-0104ca6b3b76'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:36:56.986105 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de44d50bb79d48189cd74cad8079e01f', 'request_id': 'req-158fa8ef-d701-4e67-95fc-0104ca6b3b76'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:36:56.992922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de44d50bb79d48189cd74cad8079e01f', 'request_id': 'req-158fa8ef-d701-4e67-95fc-0104ca6b3b76'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:36:57.004981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] The token cache is being invalidate because project de44d50bb79d48189cd74cad8079e01f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:36:57.005263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:36:57.005893 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-158fa8ef-d701-4e67-95fc-0104ca6b3b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:36:57.007353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1134/2306] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:36:56 2022] DELETE /identity/v3/projects/de44d50bb79d48189cd74cad8079e01f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:36:57.012195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bd5d5-fa03-468c-ac37-fef118298221 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:57.012583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bd5d5-fa03-468c-ac37-fef118298221 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:57.012916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bd5d5-fa03-468c-ac37-fef118298221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:57.033742 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-aa1bd5d5-fa03-468c-ac37-fef118298221 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:36:57.035197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-aa1bd5d5-fa03-468c-ac37-fef118298221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:36:57.092086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1173/2307] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:36:57.100796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca08db9-d26d-431c-ae61-d5e3dfe647a1 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:57.101077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca08db9-d26d-431c-ae61-d5e3dfe647a1 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:57.101349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca08db9-d26d-431c-ae61-d5e3dfe647a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:57.120361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5ca08db9-d26d-431c-ae61-d5e3dfe647a1 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:36:57.121584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5ca08db9-d26d-431c-ae61-d5e3dfe647a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:36:57.129182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1135/2308] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:36:57.133940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1bd5d5-fa03-468c-ac37-fef118298221 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:57.143547 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1bd5d5-fa03-468c-ac37-fef118298221 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:57.148892 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4dae1-800a-40a1-8fd7-9de818a5d830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:57.149496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4dae1-800a-40a1-8fd7-9de818a5d830 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:57.149754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4dae1-800a-40a1-8fd7-9de818a5d830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:57.150020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4dae1-800a-40a1-8fd7-9de818a5d830 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:57.150659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4dae1-800a-40a1-8fd7-9de818a5d830 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:36:57.150954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4dae1-800a-40a1-8fd7-9de818a5d830 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:57.152557 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4dae1-800a-40a1-8fd7-9de818a5d830 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:57.154021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:36:57.167903 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1174/2309] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:36:57.172957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca08db9-d26d-431c-ae61-d5e3dfe647a1 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:57.182483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca08db9-d26d-431c-ae61-d5e3dfe647a1 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:57.187148 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a62ca8-3eb6-4365-91bc-2879e4dcb058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:57.187734 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a62ca8-3eb6-4365-91bc-2879e4dcb058 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:57.188009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a62ca8-3eb6-4365-91bc-2879e4dcb058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:57.188278 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a62ca8-3eb6-4365-91bc-2879e4dcb058 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:57.188934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a62ca8-3eb6-4365-91bc-2879e4dcb058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:57.190824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a62ca8-3eb6-4365-91bc-2879e4dcb058 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:57.222106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1136/2310] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:36:57 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:36:57.223343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4dae1-800a-40a1-8fd7-9de818a5d830 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:57.233716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4dae1-800a-40a1-8fd7-9de818a5d830 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:57.290410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1386d1-adb6-4b37-a42b-13e8c4722d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:57.291018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1386d1-adb6-4b37-a42b-13e8c4722d4d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:57.291297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1386d1-adb6-4b37-a42b-13e8c4722d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:57.291566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1386d1-adb6-4b37-a42b-13e8c4722d4d admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:57.292183 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1386d1-adb6-4b37-a42b-13e8c4722d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:57.294217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1386d1-adb6-4b37-a42b-13e8c4722d4d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:57.314382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1175/2311] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:36:57 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:36:57.318995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a62ca8-3eb6-4365-91bc-2879e4dcb058 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:57.329150 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a62ca8-3eb6-4365-91bc-2879e4dcb058 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:57.333630 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d778b3-844c-4cc1-bf58-9a03e7364645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:57.334894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d778b3-844c-4cc1-bf58-9a03e7364645 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:57.335167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d778b3-844c-4cc1-bf58-9a03e7364645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:57.335435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d778b3-844c-4cc1-bf58-9a03e7364645 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:57.336065 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d778b3-844c-4cc1-bf58-9a03e7364645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:57.337684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d778b3-844c-4cc1-bf58-9a03e7364645 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:57.347987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1137/2312] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:36:57.352695 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1386d1-adb6-4b37-a42b-13e8c4722d4d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:57.363546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1386d1-adb6-4b37-a42b-13e8c4722d4d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:57.368197 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cb397-8b5c-4fbf-8c46-e0d31276d6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:57.368832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cb397-8b5c-4fbf-8c46-e0d31276d6dd admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:57.369289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cb397-8b5c-4fbf-8c46-e0d31276d6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:57.369750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cb397-8b5c-4fbf-8c46-e0d31276d6dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:57.370763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cb397-8b5c-4fbf-8c46-e0d31276d6dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:57.373357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cb397-8b5c-4fbf-8c46-e0d31276d6dd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:57.385552 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1176/2313] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:36:57.389791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d778b3-844c-4cc1-bf58-9a03e7364645 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:57.400391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d778b3-844c-4cc1-bf58-9a03e7364645 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:57.405331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f8c14-e2f1-4e9f-abfc-fe1a5d13552a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:57.405984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f8c14-e2f1-4e9f-abfc-fe1a5d13552a admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:57.406263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f8c14-e2f1-4e9f-abfc-fe1a5d13552a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:57.406564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f8c14-e2f1-4e9f-abfc-fe1a5d13552a admin admin] PATH_INFO: `/v3/projects/48b18efc9be64f96b2db2f2e49bda763/users/ed15542735f74f48904fc51177744f63/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:57.421224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f8c14-e2f1-4e9f-abfc-fe1a5d13552a admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b18efc9be64f96b2db2f2e49bda763, user_id=ed15542735f74f48904fc51177744f63, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:57.423865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f8c14-e2f1-4e9f-abfc-fe1a5d13552a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:57.436800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1138/2314] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:36:57 2022] PUT /identity/v3/projects/48b18efc9be64f96b2db2f2e49bda763/users/ed15542735f74f48904fc51177744f63/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:36:57.441784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49072f2-8805-4d98-9a41-fcfcd9fb6843 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:57.442092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49072f2-8805-4d98-9a41-fcfcd9fb6843 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:57.442389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49072f2-8805-4d98-9a41-fcfcd9fb6843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:57.461602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c49072f2-8805-4d98-9a41-fcfcd9fb6843 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:36:57.462780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c49072f2-8805-4d98-9a41-fcfcd9fb6843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:36:57.520499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1177/2315] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:36:57.530144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f8c14-e2f1-4e9f-abfc-fe1a5d13552a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:57.531938 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ac7f8c14-e2f1-4e9f-abfc-fe1a5d13552a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:36:57.546244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:36:57.589880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f8c14-e2f1-4e9f-abfc-fe1a5d13552a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:57.593518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a644a6a-bc2f-4340-98a4-3ccf32f0bc27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:57.594116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a644a6a-bc2f-4340-98a4-3ccf32f0bc27 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:57.594395 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a644a6a-bc2f-4340-98a4-3ccf32f0bc27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:57.594668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a644a6a-bc2f-4340-98a4-3ccf32f0bc27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:57.603466 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a644a6a-bc2f-4340-98a4-3ccf32f0bc27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:57.604936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a644a6a-bc2f-4340-98a4-3ccf32f0bc27 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:57.614094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1139/2316] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:36:58.241201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfb221-fe77-4e01-af7f-b74661454c6f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:58.241730 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfb221-fe77-4e01-af7f-b74661454c6f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:58.241730 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfb221-fe77-4e01-af7f-b74661454c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:58.268647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-30dfb221-fe77-4e01-af7f-b74661454c6f None None] MFA Rules not processed for user `ed15542735f74f48904fc51177744f63`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:36:58.269860 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-30dfb221-fe77-4e01-af7f-b74661454c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:36:58.337080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1178/2317] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:36:58 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:36:58.341720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a644a6a-bc2f-4340-98a4-3ccf32f0bc27 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:58.353335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a644a6a-bc2f-4340-98a4-3ccf32f0bc27 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:58.358679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a3a24-789d-41bb-aea7-becbf3c17ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:58.358679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a3a24-789d-41bb-aea7-becbf3c17ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:58.358955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a3a24-789d-41bb-aea7-becbf3c17ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:58.359161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a3a24-789d-41bb-aea7-becbf3c17ba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:58.359796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a3a24-789d-41bb-aea7-becbf3c17ba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:58.361772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a3a24-789d-41bb-aea7-becbf3c17ba9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:58.390579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1140/2318] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:36:58 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:36:58.394446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfb221-fe77-4e01-af7f-b74661454c6f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:58.400602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:36:58.407512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfb221-fe77-4e01-af7f-b74661454c6f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:58.462838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5585e9-e178-4884-8afe-dd2e2d2bba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:58.463439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5585e9-e178-4884-8afe-dd2e2d2bba91 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:58.463714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5585e9-e178-4884-8afe-dd2e2d2bba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:58.463983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5585e9-e178-4884-8afe-dd2e2d2bba91 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:58.464642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5585e9-e178-4884-8afe-dd2e2d2bba91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:58.466528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5585e9-e178-4884-8afe-dd2e2d2bba91 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:58.488818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1179/2319] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:36:58 2022] POST /identity/v3/users => generated 309 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:36:58.493097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a3a24-789d-41bb-aea7-becbf3c17ba9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:58.503758 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a3a24-789d-41bb-aea7-becbf3c17ba9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:58.508878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd8622-362f-4a83-b129-89a616c384d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:58.509472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd8622-362f-4a83-b129-89a616c384d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:58.509744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd8622-362f-4a83-b129-89a616c384d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:58.510010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd8622-362f-4a83-b129-89a616c384d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:58.510642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd8622-362f-4a83-b129-89a616c384d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:58.512216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd8622-362f-4a83-b129-89a616c384d9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:58.522603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1141/2320] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:36:58.526550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5585e9-e178-4884-8afe-dd2e2d2bba91 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:58.536393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5585e9-e178-4884-8afe-dd2e2d2bba91 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:58.540919 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9b65b-6ac4-41c8-85ac-f819fab12ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:58.541555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9b65b-6ac4-41c8-85ac-f819fab12ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:58.541831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9b65b-6ac4-41c8-85ac-f819fab12ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:58.542098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9b65b-6ac4-41c8-85ac-f819fab12ca3 admin admin] PATH_INFO: `/v3/projects/933f2107f5554cd59df6c3514ff12dec/users/7c2aeec45a1146fb9d7cac77fa04c6e6/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:58.556709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9b65b-6ac4-41c8-85ac-f819fab12ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=933f2107f5554cd59df6c3514ff12dec, user_id=7c2aeec45a1146fb9d7cac77fa04c6e6, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:58.559327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9b65b-6ac4-41c8-85ac-f819fab12ca3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:58.572448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1180/2321] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:36:58 2022] PUT /identity/v3/projects/933f2107f5554cd59df6c3514ff12dec/users/7c2aeec45a1146fb9d7cac77fa04c6e6/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:36:58.576344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd8622-362f-4a83-b129-89a616c384d9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:58.586018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd8622-362f-4a83-b129-89a616c384d9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:58.640518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14de112-dfb0-4779-8d9a-26969632ec2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:58.641046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14de112-dfb0-4779-8d9a-26969632ec2e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:58.641323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14de112-dfb0-4779-8d9a-26969632ec2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:58.641592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14de112-dfb0-4779-8d9a-26969632ec2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:58.642247 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14de112-dfb0-4779-8d9a-26969632ec2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:58.643811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14de112-dfb0-4779-8d9a-26969632ec2e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:58.653327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1142/2322] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:36:58.657493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9b65b-6ac4-41c8-85ac-f819fab12ca3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:36:58.668510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9b65b-6ac4-41c8-85ac-f819fab12ca3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:36:58.673170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416c6bf-5c80-44ca-b8c0-53b1f6f0fd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:36:58.673809 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416c6bf-5c80-44ca-b8c0-53b1f6f0fd1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:58.674092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416c6bf-5c80-44ca-b8c0-53b1f6f0fd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:58.674361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416c6bf-5c80-44ca-b8c0-53b1f6f0fd1e admin admin] PATH_INFO: `/v3/projects/933f2107f5554cd59df6c3514ff12dec/users/7c2aeec45a1146fb9d7cac77fa04c6e6/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:58.676249 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416c6bf-5c80-44ca-b8c0-53b1f6f0fd1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=933f2107f5554cd59df6c3514ff12dec, user_id=7c2aeec45a1146fb9d7cac77fa04c6e6, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:36:58.679576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416c6bf-5c80-44ca-b8c0-53b1f6f0fd1e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:36:58.689856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1181/2323] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:36:58 2022] PUT /identity/v3/projects/933f2107f5554cd59df6c3514ff12dec/users/7c2aeec45a1146fb9d7cac77fa04c6e6/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:36:59.560164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f07f8-3171-4a45-b86f-974b40a7898f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:36:59.560478 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f07f8-3171-4a45-b86f-974b40a7898f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:36:59.560757 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f07f8-3171-4a45-b86f-974b40a7898f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:36:59.588672 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e86f07f8-3171-4a45-b86f-974b40a7898f None None] MFA Rules not processed for user `7c2aeec45a1146fb9d7cac77fa04c6e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:36:59.589869 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e86f07f8-3171-4a45-b86f-974b40a7898f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:36:59.659295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1143/2324] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:36:59 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:00.662152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416c6bf-5c80-44ca-b8c0-53b1f6f0fd1e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:00.723708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416c6bf-5c80-44ca-b8c0-53b1f6f0fd1e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:00.728228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:00.728885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:00.729160 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:00.729427 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] PATH_INFO: `/v3/users/ed15542735f74f48904fc51177744f63` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:00.730518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed15542735f74f48904fc51177744f63)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:00.732439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:00.779424 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed15542735f74f48904fc51177744f63'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:00.786655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed15542735f74f48904fc51177744f63'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:00.787312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed15542735f74f48904fc51177744f63'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:00.788957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:00.795272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed15542735f74f48904fc51177744f63'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:00.808581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed15542735f74f48904fc51177744f63'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:00.814213 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1182/2325] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:00 2022] DELETE /identity/v3/users/ed15542735f74f48904fc51177744f63 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:00.982805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f07f8-3171-4a45-b86f-974b40a7898f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:00.989485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:00.995041 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f07f8-3171-4a45-b86f-974b40a7898f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:01.072717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:01.073276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:01.073554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:01.073824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] PATH_INFO: `/v3/projects/48b18efc9be64f96b2db2f2e49bda763` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:01.075162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48b18efc9be64f96b2db2f2e49bda763)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:01.077117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:01.127868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48b18efc9be64f96b2db2f2e49bda763', 'request_id': 'req-92e99106-82b5-454d-8168-6f1688a9c439'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.133484 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48b18efc9be64f96b2db2f2e49bda763', 'request_id': 'req-92e99106-82b5-454d-8168-6f1688a9c439'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.139693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48b18efc9be64f96b2db2f2e49bda763', 'request_id': 'req-92e99106-82b5-454d-8168-6f1688a9c439'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.153358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] The token cache is being invalidate because project 48b18efc9be64f96b2db2f2e49bda763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:01.153646 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.154266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.155737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1144/2326] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:00 2022] DELETE /identity/v3/projects/48b18efc9be64f96b2db2f2e49bda763 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:01.160039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:01.161536 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:01.175400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c9f57-a0a1-48ad-be50-2952d7819b39 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:01.233713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc9af7-6d3d-4739-8751-068c6c338fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:01.234353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc9af7-6d3d-4739-8751-068c6c338fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:01.234629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc9af7-6d3d-4739-8751-068c6c338fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:01.234898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc9af7-6d3d-4739-8751-068c6c338fe8 admin admin] PATH_INFO: `/v3/users/7c2aeec45a1146fb9d7cac77fa04c6e6` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:01.236011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc9af7-6d3d-4739-8751-068c6c338fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2aeec45a1146fb9d7cac77fa04c6e6)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:01.237943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc9af7-6d3d-4739-8751-068c6c338fe8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:01.290479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a1dc9af7-6d3d-4739-8751-068c6c338fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2aeec45a1146fb9d7cac77fa04c6e6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.297405 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a1dc9af7-6d3d-4739-8751-068c6c338fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2aeec45a1146fb9d7cac77fa04c6e6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.297995 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a1dc9af7-6d3d-4739-8751-068c6c338fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2aeec45a1146fb9d7cac77fa04c6e6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.305659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a1dc9af7-6d3d-4739-8751-068c6c338fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2aeec45a1146fb9d7cac77fa04c6e6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.317933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a1dc9af7-6d3d-4739-8751-068c6c338fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2aeec45a1146fb9d7cac77fa04c6e6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.323754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1183/2327] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:01 2022] DELETE /identity/v3/users/7c2aeec45a1146fb9d7cac77fa04c6e6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:37:01.505057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:01.506713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:01.520345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e99106-82b5-454d-8168-6f1688a9c439 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:01.577304 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:01.577864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:01.578173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:01.578439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] PATH_INFO: `/v3/projects/933f2107f5554cd59df6c3514ff12dec` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:01.579567 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=933f2107f5554cd59df6c3514ff12dec)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:01.581519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:01.617787 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:01.628631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '933f2107f5554cd59df6c3514ff12dec', 'request_id': 'req-cd6a5719-0289-417a-b210-2c7259066525'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.634680 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '933f2107f5554cd59df6c3514ff12dec', 'request_id': 'req-cd6a5719-0289-417a-b210-2c7259066525'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.640862 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '933f2107f5554cd59df6c3514ff12dec', 'request_id': 'req-cd6a5719-0289-417a-b210-2c7259066525'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.653470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] The token cache is being invalidate because project 933f2107f5554cd59df6c3514ff12dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:01.653754 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.654387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cd6a5719-0289-417a-b210-2c7259066525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:01.655843 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1145/2328] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:01 2022] DELETE /identity/v3/projects/933f2107f5554cd59df6c3514ff12dec => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:37:01.661026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c470d39-51bc-48bf-aa3e-661e08d19be3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:01.661313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c470d39-51bc-48bf-aa3e-661e08d19be3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:01.661586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c470d39-51bc-48bf-aa3e-661e08d19be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:01.680771 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0c470d39-51bc-48bf-aa3e-661e08d19be3 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:01.681940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0c470d39-51bc-48bf-aa3e-661e08d19be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:01.728560 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:01.742305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1184/2329] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:01.751161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d003b-ea25-4184-b906-d15bdced215d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:01.751441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d003b-ea25-4184-b906-d15bdced215d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:01.751714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d003b-ea25-4184-b906-d15bdced215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:01.771829 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8f4d003b-ea25-4184-b906-d15bdced215d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:01.773058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8f4d003b-ea25-4184-b906-d15bdced215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:01.780914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1146/2330] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:01.785078 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c470d39-51bc-48bf-aa3e-661e08d19be3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:01.794725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c470d39-51bc-48bf-aa3e-661e08d19be3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:01.799254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475e94c-6831-4ed9-8a12-d18ac5eeb202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:01.799846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475e94c-6831-4ed9-8a12-d18ac5eeb202 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:01.800124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475e94c-6831-4ed9-8a12-d18ac5eeb202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:01.800416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475e94c-6831-4ed9-8a12-d18ac5eeb202 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:01.801064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475e94c-6831-4ed9-8a12-d18ac5eeb202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:37:01.801364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475e94c-6831-4ed9-8a12-d18ac5eeb202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:01.802976 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475e94c-6831-4ed9-8a12-d18ac5eeb202 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:01.814250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1185/2331] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:37:01.818871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d003b-ea25-4184-b906-d15bdced215d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:01.827940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d003b-ea25-4184-b906-d15bdced215d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:01.832636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea3815-8afe-473a-bf83-bb6108579898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:01.833227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3815-8afe-473a-bf83-bb6108579898 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:01.833508 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3815-8afe-473a-bf83-bb6108579898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:01.834370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3815-8afe-473a-bf83-bb6108579898 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:01.835004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea3815-8afe-473a-bf83-bb6108579898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:01.836942 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea3815-8afe-473a-bf83-bb6108579898 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:01.863010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1147/2332] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:37:01 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:37:01.867059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475e94c-6831-4ed9-8a12-d18ac5eeb202 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:01.876982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475e94c-6831-4ed9-8a12-d18ac5eeb202 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:01.932691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede387aa-dcd7-468b-bee3-e086a5caf1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:01.933284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede387aa-dcd7-468b-bee3-e086a5caf1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:01.933562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede387aa-dcd7-468b-bee3-e086a5caf1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:01.933833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede387aa-dcd7-468b-bee3-e086a5caf1b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:01.934457 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede387aa-dcd7-468b-bee3-e086a5caf1b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:01.936398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede387aa-dcd7-468b-bee3-e086a5caf1b7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:01.956977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1186/2333] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:37:01 2022] POST /identity/v3/users => generated 282 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:37:01.961176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea3815-8afe-473a-bf83-bb6108579898 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:01.971154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea3815-8afe-473a-bf83-bb6108579898 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:01.975695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9283e0-0413-4938-99d9-421711877350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:01.976282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9283e0-0413-4938-99d9-421711877350 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:01.976590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9283e0-0413-4938-99d9-421711877350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:01.976860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9283e0-0413-4938-99d9-421711877350 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:01.977493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9283e0-0413-4938-99d9-421711877350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:01.979058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9283e0-0413-4938-99d9-421711877350 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:01.989035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1148/2334] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:01.993113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede387aa-dcd7-468b-bee3-e086a5caf1b7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:02.002886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede387aa-dcd7-468b-bee3-e086a5caf1b7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:02.008323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e9c06-88ae-45e0-a7cb-0b5c79775092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:02.008940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9c06-88ae-45e0-a7cb-0b5c79775092 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:02.009219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9c06-88ae-45e0-a7cb-0b5c79775092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:02.009492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9c06-88ae-45e0-a7cb-0b5c79775092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:02.010125 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e9c06-88ae-45e0-a7cb-0b5c79775092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:02.011889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e9c06-88ae-45e0-a7cb-0b5c79775092 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:02.022234 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1187/2335] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:02.026173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9283e0-0413-4938-99d9-421711877350 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:02.036055 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9283e0-0413-4938-99d9-421711877350 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:02.040661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bb6e-f198-4ebd-8658-d1eff8b649a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:02.041291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bb6e-f198-4ebd-8658-d1eff8b649a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:02.041563 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bb6e-f198-4ebd-8658-d1eff8b649a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:02.041833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86bb6e-f198-4ebd-8658-d1eff8b649a8 admin admin] PATH_INFO: `/v3/projects/d8f234f9ecfe4b02932dcb7ecf7927d2/users/60af6365c49a4cbca0fabc6b3c527716/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:02.058113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bb6e-f198-4ebd-8658-d1eff8b649a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f234f9ecfe4b02932dcb7ecf7927d2, user_id=60af6365c49a4cbca0fabc6b3c527716, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:02.060811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86bb6e-f198-4ebd-8658-d1eff8b649a8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:02.074408 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1149/2336] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:02 2022] PUT /identity/v3/projects/d8f234f9ecfe4b02932dcb7ecf7927d2/users/60af6365c49a4cbca0fabc6b3c527716/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:02.084656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd0630-e07c-4a1d-aca3-57534f718483 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:02.084965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd0630-e07c-4a1d-aca3-57534f718483 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:02.085244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd0630-e07c-4a1d-aca3-57534f718483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:02.112682 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a2bd0630-e07c-4a1d-aca3-57534f718483 None None] MFA Rules not processed for user `60af6365c49a4cbca0fabc6b3c527716`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:02.113888 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a2bd0630-e07c-4a1d-aca3-57534f718483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:02.185853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1188/2337] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:02.190316 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bb6e-f198-4ebd-8658-d1eff8b649a8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:02.249528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86bb6e-f198-4ebd-8658-d1eff8b649a8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:02.254142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788599b4-cb16-4e75-a73e-20d511c66d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:02.254760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788599b4-cb16-4e75-a73e-20d511c66d59 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:02.255023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788599b4-cb16-4e75-a73e-20d511c66d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:02.255294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788599b4-cb16-4e75-a73e-20d511c66d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:02.256587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788599b4-cb16-4e75-a73e-20d511c66d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:02.258488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788599b4-cb16-4e75-a73e-20d511c66d59 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:02.284001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1150/2338] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:37:02 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:02.288233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd0630-e07c-4a1d-aca3-57534f718483 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:02.298795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd0630-e07c-4a1d-aca3-57534f718483 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:02.342352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:02.355008 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90c13-44b0-4b09-b1c5-2d92e2cab801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:02.355609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90c13-44b0-4b09-b1c5-2d92e2cab801 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:02.355884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90c13-44b0-4b09-b1c5-2d92e2cab801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:02.356156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90c13-44b0-4b09-b1c5-2d92e2cab801 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:02.356816 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90c13-44b0-4b09-b1c5-2d92e2cab801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:02.358716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90c13-44b0-4b09-b1c5-2d92e2cab801 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:02.384603 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1189/2339] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:37:02 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:02.385926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788599b4-cb16-4e75-a73e-20d511c66d59 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:02.400748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788599b4-cb16-4e75-a73e-20d511c66d59 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:02.403435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04008aab-8b4a-4023-9e4e-b55e13166bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:02.404064 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04008aab-8b4a-4023-9e4e-b55e13166bbe admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:02.404339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04008aab-8b4a-4023-9e4e-b55e13166bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:02.404647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04008aab-8b4a-4023-9e4e-b55e13166bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:02.405287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04008aab-8b4a-4023-9e4e-b55e13166bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:02.406873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04008aab-8b4a-4023-9e4e-b55e13166bbe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:02.418576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1151/2340] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:02.422847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90c13-44b0-4b09-b1c5-2d92e2cab801 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:02.434286 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90c13-44b0-4b09-b1c5-2d92e2cab801 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:02.438997 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21defb-b743-4678-aea4-0dc00c48e1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:02.439640 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21defb-b743-4678-aea4-0dc00c48e1cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:02.439914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21defb-b743-4678-aea4-0dc00c48e1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:02.440180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21defb-b743-4678-aea4-0dc00c48e1cc admin admin] PATH_INFO: `/v3/projects/d8d97edf0fa648bba2937342ee38071c/users/99a034912adc449fb7d0244ff57fbc43/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:02.455604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21defb-b743-4678-aea4-0dc00c48e1cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d97edf0fa648bba2937342ee38071c, user_id=99a034912adc449fb7d0244ff57fbc43, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:02.458319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21defb-b743-4678-aea4-0dc00c48e1cc admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:02.471754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1190/2341] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:02 2022] PUT /identity/v3/projects/d8d97edf0fa648bba2937342ee38071c/users/99a034912adc449fb7d0244ff57fbc43/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:37:02.476346 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04008aab-8b4a-4023-9e4e-b55e13166bbe admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:02.486964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04008aab-8b4a-4023-9e4e-b55e13166bbe admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:02.520290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:02.548318 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda26ffc-7880-4666-931c-f235f79af44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:02.548953 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda26ffc-7880-4666-931c-f235f79af44d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:02.549231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda26ffc-7880-4666-931c-f235f79af44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:02.549501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda26ffc-7880-4666-931c-f235f79af44d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:02.550141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda26ffc-7880-4666-931c-f235f79af44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:02.551768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda26ffc-7880-4666-931c-f235f79af44d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:02.562717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1152/2342] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:02.566790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21defb-b743-4678-aea4-0dc00c48e1cc admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:02.577050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21defb-b743-4678-aea4-0dc00c48e1cc admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:02.581569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdbded-0a42-414c-8943-58947f312b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:02.582211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdbded-0a42-414c-8943-58947f312b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:02.582485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdbded-0a42-414c-8943-58947f312b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:02.582753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdbded-0a42-414c-8943-58947f312b2d admin admin] PATH_INFO: `/v3/projects/d8d97edf0fa648bba2937342ee38071c/users/99a034912adc449fb7d0244ff57fbc43/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:02.584637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afdbded-0a42-414c-8943-58947f312b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d97edf0fa648bba2937342ee38071c, user_id=99a034912adc449fb7d0244ff57fbc43, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:02.587241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afdbded-0a42-414c-8943-58947f312b2d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:02.597548 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1191/2343] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:02 2022] PUT /identity/v3/projects/d8d97edf0fa648bba2937342ee38071c/users/99a034912adc449fb7d0244ff57fbc43/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:02.609482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc89089-50c3-40dc-8c41-cf936050df1a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:02.609770 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc89089-50c3-40dc-8c41-cf936050df1a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:02.610045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc89089-50c3-40dc-8c41-cf936050df1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:02.638239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-bcc89089-50c3-40dc-8c41-cf936050df1a None None] MFA Rules not processed for user `99a034912adc449fb7d0244ff57fbc43`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:02.639432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-bcc89089-50c3-40dc-8c41-cf936050df1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:02.705305 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1153/2344] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:02.717313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdbded-0a42-414c-8943-58947f312b2d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:02.718982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3afdbded-0a42-414c-8943-58947f312b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:02.777728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdbded-0a42-414c-8943-58947f312b2d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:02.781410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0085d-f48d-4bf9-b1ad-374a27090bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:02.782018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0085d-f48d-4bf9-b1ad-374a27090bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:02.783848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0085d-f48d-4bf9-b1ad-374a27090bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:02.783848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0085d-f48d-4bf9-b1ad-374a27090bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:02.792822 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0085d-f48d-4bf9-b1ad-374a27090bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:02.794256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0085d-f48d-4bf9-b1ad-374a27090bb7 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:02.803818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1192/2345] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:03.222315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc89089-50c3-40dc-8c41-cf936050df1a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:03.286033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc89089-50c3-40dc-8c41-cf936050df1a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:03.298366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:03.299761 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:03.300043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:03.300313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] PATH_INFO: `/v3/users/60af6365c49a4cbca0fabc6b3c527716` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:03.302080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60af6365c49a4cbca0fabc6b3c527716)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:03.305118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:03.355411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60af6365c49a4cbca0fabc6b3c527716'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:03.363513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60af6365c49a4cbca0fabc6b3c527716'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:03.364097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60af6365c49a4cbca0fabc6b3c527716'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:03.367150 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:03.372666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60af6365c49a4cbca0fabc6b3c527716'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:03.387995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60af6365c49a4cbca0fabc6b3c527716'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:03.394154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1154/2346] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:03 2022] DELETE /identity/v3/users/60af6365c49a4cbca0fabc6b3c527716 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:03.404000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0106fa-80cf-4d93-8308-2346f4e33e5d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:03.404000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0106fa-80cf-4d93-8308-2346f4e33e5d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:03.404212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0106fa-80cf-4d93-8308-2346f4e33e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:03.424637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3d0106fa-80cf-4d93-8308-2346f4e33e5d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:03.425943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3d0106fa-80cf-4d93-8308-2346f4e33e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:03.484384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1193/2347] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:03.494616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:03.496093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:03.579955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfa246-19f2-484f-a6b0-e2a1bdc156b4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:03.586324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5397e9d-a53d-441c-b15a-2794c793f238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:03.586962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5397e9d-a53d-441c-b15a-2794c793f238 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:03.587247 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5397e9d-a53d-441c-b15a-2794c793f238 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:03.587515 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5397e9d-a53d-441c-b15a-2794c793f238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:03.611796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5397e9d-a53d-441c-b15a-2794c793f238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:03.613932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5397e9d-a53d-441c-b15a-2794c793f238 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:03.628223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1155/2348] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:03.960092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0106fa-80cf-4d93-8308-2346f4e33e5d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:03.962154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3d0106fa-80cf-4d93-8308-2346f4e33e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:03.974462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0106fa-80cf-4d93-8308-2346f4e33e5d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:03.979096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:03.979791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:03.980137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:03.980506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] PATH_INFO: `/v3/projects/d8f234f9ecfe4b02932dcb7ecf7927d2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:03.981681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f234f9ecfe4b02932dcb7ecf7927d2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:03.983655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:04.026287 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:04.034826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f234f9ecfe4b02932dcb7ecf7927d2', 'request_id': 'req-a8d1e8fe-3126-47c9-a886-e994d12bb769'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.041777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f234f9ecfe4b02932dcb7ecf7927d2', 'request_id': 'req-a8d1e8fe-3126-47c9-a886-e994d12bb769'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.048319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f234f9ecfe4b02932dcb7ecf7927d2', 'request_id': 'req-a8d1e8fe-3126-47c9-a886-e994d12bb769'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.061190 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] The token cache is being invalidate because project d8f234f9ecfe4b02932dcb7ecf7927d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:04.061475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.062081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.063915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1194/2349] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:03 2022] DELETE /identity/v3/projects/d8f234f9ecfe4b02932dcb7ecf7927d2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:04.067800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5397e9d-a53d-441c-b15a-2794c793f238 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:04.085808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5397e9d-a53d-441c-b15a-2794c793f238 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:04.142920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f114e9-855d-416e-baea-b7524222458a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:04.143617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f114e9-855d-416e-baea-b7524222458a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:04.144065 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f114e9-855d-416e-baea-b7524222458a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:04.144341 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f114e9-855d-416e-baea-b7524222458a admin admin] PATH_INFO: `/v3/users/99a034912adc449fb7d0244ff57fbc43` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:04.145487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f114e9-855d-416e-baea-b7524222458a admin admin] RBAC: Authorizing `identity:delete_user(user_id=99a034912adc449fb7d0244ff57fbc43)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:04.147981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f114e9-855d-416e-baea-b7524222458a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:04.198456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-80f114e9-855d-416e-baea-b7524222458a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99a034912adc449fb7d0244ff57fbc43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.205764 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-80f114e9-855d-416e-baea-b7524222458a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99a034912adc449fb7d0244ff57fbc43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.206440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-80f114e9-855d-416e-baea-b7524222458a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99a034912adc449fb7d0244ff57fbc43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.216050 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-80f114e9-855d-416e-baea-b7524222458a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99a034912adc449fb7d0244ff57fbc43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.230937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-80f114e9-855d-416e-baea-b7524222458a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99a034912adc449fb7d0244ff57fbc43'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.232946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:04.239284 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1156/2350] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:04 2022] DELETE /identity/v3/users/99a034912adc449fb7d0244ff57fbc43 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:04.588247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:04.589767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:04.641196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1e8fe-3126-47c9-a886-e994d12bb769 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:04.744040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:04.744714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:04.744990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:04.745261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] PATH_INFO: `/v3/projects/d8d97edf0fa648bba2937342ee38071c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:04.746396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d97edf0fa648bba2937342ee38071c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:04.748329 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:04.796218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d97edf0fa648bba2937342ee38071c', 'request_id': 'req-c41133d4-60d1-47e3-b689-6b713c6d02e8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.801762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d97edf0fa648bba2937342ee38071c', 'request_id': 'req-c41133d4-60d1-47e3-b689-6b713c6d02e8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.808085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d97edf0fa648bba2937342ee38071c', 'request_id': 'req-c41133d4-60d1-47e3-b689-6b713c6d02e8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.811809 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:04.823568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] The token cache is being invalidate because project d8d97edf0fa648bba2937342ee38071c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:04.823746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.824435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-c41133d4-60d1-47e3-b689-6b713c6d02e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:04.826142 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1195/2351] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:04 2022] DELETE /identity/v3/projects/d8d97edf0fa648bba2937342ee38071c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:04.831616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959c8e5-bc28-44f4-8cc7-8ea61a392f26 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:04.831902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959c8e5-bc28-44f4-8cc7-8ea61a392f26 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:04.832227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959c8e5-bc28-44f4-8cc7-8ea61a392f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:04.900321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2959c8e5-bc28-44f4-8cc7-8ea61a392f26 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:04.901661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2959c8e5-bc28-44f4-8cc7-8ea61a392f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:04.961025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1157/2352] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:04.968816 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e6512-8d50-434d-9d6e-35da8541598a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:04.969330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e6512-8d50-434d-9d6e-35da8541598a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:04.969330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e6512-8d50-434d-9d6e-35da8541598a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:04.988482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c28e6512-8d50-434d-9d6e-35da8541598a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:04.989660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c28e6512-8d50-434d-9d6e-35da8541598a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:05.000747 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1196/2353] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:05.003706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959c8e5-bc28-44f4-8cc7-8ea61a392f26 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.015763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959c8e5-bc28-44f4-8cc7-8ea61a392f26 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.034163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fa988-ee56-4b9e-9b92-f787759f18a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.035114 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fa988-ee56-4b9e-9b92-f787759f18a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.035455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fa988-ee56-4b9e-9b92-f787759f18a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.035792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fa988-ee56-4b9e-9b92-f787759f18a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.036624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fa988-ee56-4b9e-9b92-f787759f18a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:37:05.037077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fa988-ee56-4b9e-9b92-f787759f18a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.038760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fa988-ee56-4b9e-9b92-f787759f18a1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:05.053616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1158/2354] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:37:05.057987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e6512-8d50-434d-9d6e-35da8541598a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.067232 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e6512-8d50-434d-9d6e-35da8541598a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.072458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8e869-d596-42bf-8d55-54f8c9262237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.073087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8e869-d596-42bf-8d55-54f8c9262237 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.073333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8e869-d596-42bf-8d55-54f8c9262237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.073603 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8e869-d596-42bf-8d55-54f8c9262237 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.074233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8e869-d596-42bf-8d55-54f8c9262237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.076142 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8e869-d596-42bf-8d55-54f8c9262237 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:05.103327 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1197/2355] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:37:05 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:37:05.107548 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fa988-ee56-4b9e-9b92-f787759f18a1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.118643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fa988-ee56-4b9e-9b92-f787759f18a1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.178927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9c951-8499-46e9-a961-1a2592445e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.179537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9c951-8499-46e9-a961-1a2592445e0c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.179840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9c951-8499-46e9-a961-1a2592445e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.180109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9c951-8499-46e9-a961-1a2592445e0c admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.180772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9c951-8499-46e9-a961-1a2592445e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.182712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9c951-8499-46e9-a961-1a2592445e0c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:05.209036 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1159/2356] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:37:05 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:05.210222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8e869-d596-42bf-8d55-54f8c9262237 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.220214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8e869-d596-42bf-8d55-54f8c9262237 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.224821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd118bc6-e167-44d8-9b79-6ea4379251a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.225422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd118bc6-e167-44d8-9b79-6ea4379251a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.225699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd118bc6-e167-44d8-9b79-6ea4379251a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.225972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd118bc6-e167-44d8-9b79-6ea4379251a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.226610 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd118bc6-e167-44d8-9b79-6ea4379251a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.228192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd118bc6-e167-44d8-9b79-6ea4379251a1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:05.238396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1198/2357] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:05.242967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9c951-8499-46e9-a961-1a2592445e0c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.253360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9c951-8499-46e9-a961-1a2592445e0c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.258883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e633b7-8fb2-4aa0-b528-0dc155779f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.259477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e633b7-8fb2-4aa0-b528-0dc155779f76 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.259749 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e633b7-8fb2-4aa0-b528-0dc155779f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.260036 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e633b7-8fb2-4aa0-b528-0dc155779f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.260711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e633b7-8fb2-4aa0-b528-0dc155779f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.262378 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e633b7-8fb2-4aa0-b528-0dc155779f76 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:05.273295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1160/2358] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:37:05.277325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd118bc6-e167-44d8-9b79-6ea4379251a1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.287265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd118bc6-e167-44d8-9b79-6ea4379251a1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.292264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5e3e4-61a5-4101-841f-8605c70366d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.293034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5e3e4-61a5-4101-841f-8605c70366d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.293385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5e3e4-61a5-4101-841f-8605c70366d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.293719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5e3e4-61a5-4101-841f-8605c70366d3 admin admin] PATH_INFO: `/v3/projects/0a12aae8733e4adbba33c0fdd291b60f/users/43d9e1179c9944ac8179c3e1ee66be6e/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.309875 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5e3e4-61a5-4101-841f-8605c70366d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a12aae8733e4adbba33c0fdd291b60f, user_id=43d9e1179c9944ac8179c3e1ee66be6e, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.312675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5e3e4-61a5-4101-841f-8605c70366d3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:05.326509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1199/2359] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:05 2022] PUT /identity/v3/projects/0a12aae8733e4adbba33c0fdd291b60f/users/43d9e1179c9944ac8179c3e1ee66be6e/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:05.337925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e793f-455c-4338-8402-0de32b688d02 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.338233 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e793f-455c-4338-8402-0de32b688d02 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.338553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e793f-455c-4338-8402-0de32b688d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.367781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-271e793f-455c-4338-8402-0de32b688d02 None None] MFA Rules not processed for user `43d9e1179c9944ac8179c3e1ee66be6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:05.369027 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-271e793f-455c-4338-8402-0de32b688d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:05.400752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:05.455921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1161/2360] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:05.460477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5e3e4-61a5-4101-841f-8605c70366d3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.530040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5e3e4-61a5-4101-841f-8605c70366d3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.534993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f280d94-199f-4420-9efc-0338841b00d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.535594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f280d94-199f-4420-9efc-0338841b00d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.535872 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f280d94-199f-4420-9efc-0338841b00d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.536138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f280d94-199f-4420-9efc-0338841b00d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.537320 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f280d94-199f-4420-9efc-0338841b00d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.539211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f280d94-199f-4420-9efc-0338841b00d6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:05.567992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1200/2361] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:37:05 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:37:05.572002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e793f-455c-4338-8402-0de32b688d02 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.582781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e793f-455c-4338-8402-0de32b688d02 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.642673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93725a1-4dc1-44a3-a665-2b0a1db3744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.643294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93725a1-4dc1-44a3-a665-2b0a1db3744e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.643560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93725a1-4dc1-44a3-a665-2b0a1db3744e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.643959 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93725a1-4dc1-44a3-a665-2b0a1db3744e admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.644900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93725a1-4dc1-44a3-a665-2b0a1db3744e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.647101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93725a1-4dc1-44a3-a665-2b0a1db3744e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:05.670598 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1162/2362] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:37:05 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:05.674570 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f280d94-199f-4420-9efc-0338841b00d6 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.685878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f280d94-199f-4420-9efc-0338841b00d6 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.691186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428ae57-c5b8-49a0-8569-ba868f1c360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.691779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428ae57-c5b8-49a0-8569-ba868f1c360f admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.692057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428ae57-c5b8-49a0-8569-ba868f1c360f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.692321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428ae57-c5b8-49a0-8569-ba868f1c360f admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.693061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428ae57-c5b8-49a0-8569-ba868f1c360f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.694693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428ae57-c5b8-49a0-8569-ba868f1c360f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:05.710398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1201/2363] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:37:05.711536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93725a1-4dc1-44a3-a665-2b0a1db3744e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.721501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93725a1-4dc1-44a3-a665-2b0a1db3744e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.726690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0154b4-a044-419f-90f1-5561b35d8014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.727281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0154b4-a044-419f-90f1-5561b35d8014 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.727554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0154b4-a044-419f-90f1-5561b35d8014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.727885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0154b4-a044-419f-90f1-5561b35d8014 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.728563 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0154b4-a044-419f-90f1-5561b35d8014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.730156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0154b4-a044-419f-90f1-5561b35d8014 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:05.741676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1163/2364] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:37:05.745992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428ae57-c5b8-49a0-8569-ba868f1c360f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.756088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428ae57-c5b8-49a0-8569-ba868f1c360f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.760642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95d51a-b6c6-4ed5-a9b3-a7753f18ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.761285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95d51a-b6c6-4ed5-a9b3-a7753f18ff89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.761695 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95d51a-b6c6-4ed5-a9b3-a7753f18ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.761822 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95d51a-b6c6-4ed5-a9b3-a7753f18ff89 admin admin] PATH_INFO: `/v3/projects/426aa955e6154fdd89c6c6d3401b117e/users/2cac818f99de48c79c20186558e66f8a/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.776808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95d51a-b6c6-4ed5-a9b3-a7753f18ff89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=426aa955e6154fdd89c6c6d3401b117e, user_id=2cac818f99de48c79c20186558e66f8a, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.779376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95d51a-b6c6-4ed5-a9b3-a7753f18ff89 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:05.793840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1202/2365] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:05 2022] PUT /identity/v3/projects/426aa955e6154fdd89c6c6d3401b117e/users/2cac818f99de48c79c20186558e66f8a/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:05.810700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5d08-256a-4ccc-a9ea-4739dbb8a12e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.810988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5d08-256a-4ccc-a9ea-4739dbb8a12e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.811255 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5d08-256a-4ccc-a9ea-4739dbb8a12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.845778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1e5c5d08-256a-4ccc-a9ea-4739dbb8a12e None None] MFA Rules not processed for user `2cac818f99de48c79c20186558e66f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:05.847069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1e5c5d08-256a-4ccc-a9ea-4739dbb8a12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:05.914827 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1164/2366] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:05.919236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95d51a-b6c6-4ed5-a9b3-a7753f18ff89 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:05.967916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:05.989390 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95d51a-b6c6-4ed5-a9b3-a7753f18ff89 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:05.994227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558d2a2-995f-446b-a037-37e2ad30ffe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:05.994826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558d2a2-995f-446b-a037-37e2ad30ffe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:05.995098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558d2a2-995f-446b-a037-37e2ad30ffe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:05.995361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558d2a2-995f-446b-a037-37e2ad30ffe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:05.995972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558d2a2-995f-446b-a037-37e2ad30ffe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:05.998348 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558d2a2-995f-446b-a037-37e2ad30ffe4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:06.026027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1203/2367] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:37:05 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:06.030289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c5d08-256a-4ccc-a9ea-4739dbb8a12e None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:06.041755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c5d08-256a-4ccc-a9ea-4739dbb8a12e None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:06.070902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:06.100543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec482dc-9ef7-475d-a3ec-331a2c0fdeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:06.101327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec482dc-9ef7-475d-a3ec-331a2c0fdeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:06.101745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec482dc-9ef7-475d-a3ec-331a2c0fdeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:06.102026 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec482dc-9ef7-475d-a3ec-331a2c0fdeb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:06.103151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec482dc-9ef7-475d-a3ec-331a2c0fdeb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:06.105589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec482dc-9ef7-475d-a3ec-331a2c0fdeb2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:06.128438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1165/2368] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:37:06 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:06.133498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558d2a2-995f-446b-a037-37e2ad30ffe4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:06.143936 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558d2a2-995f-446b-a037-37e2ad30ffe4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:06.149396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d771ab-66ca-44aa-8633-af4e267ab5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:06.150002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d771ab-66ca-44aa-8633-af4e267ab5eb admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:06.150282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d771ab-66ca-44aa-8633-af4e267ab5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:06.150558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d771ab-66ca-44aa-8633-af4e267ab5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:06.151194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d771ab-66ca-44aa-8633-af4e267ab5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:06.152818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d771ab-66ca-44aa-8633-af4e267ab5eb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:06.168216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1204/2369] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:06.172589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec482dc-9ef7-475d-a3ec-331a2c0fdeb2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:06.182799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec482dc-9ef7-475d-a3ec-331a2c0fdeb2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:06.187760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9df2d-e4e0-43f9-8f8f-8e1d1ebf6eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:06.188435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9df2d-e4e0-43f9-8f8f-8e1d1ebf6eca admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:06.189033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9df2d-e4e0-43f9-8f8f-8e1d1ebf6eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:06.189033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9df2d-e4e0-43f9-8f8f-8e1d1ebf6eca admin admin] PATH_INFO: `/v3/projects/4220bef6f7c5474b987bb3448a753754/users/9680b745737546e994bb51c8b6e2d6d4/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:06.204075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9df2d-e4e0-43f9-8f8f-8e1d1ebf6eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=4220bef6f7c5474b987bb3448a753754, user_id=9680b745737546e994bb51c8b6e2d6d4, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:06.206833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9df2d-e4e0-43f9-8f8f-8e1d1ebf6eca admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:06.220348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1166/2370] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:06 2022] PUT /identity/v3/projects/4220bef6f7c5474b987bb3448a753754/users/9680b745737546e994bb51c8b6e2d6d4/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:06.224344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d771ab-66ca-44aa-8633-af4e267ab5eb admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:06.234701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d771ab-66ca-44aa-8633-af4e267ab5eb admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:06.294096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c56d9-b611-445a-9c13-9c4d69c5498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:06.294974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c56d9-b611-445a-9c13-9c4d69c5498c admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:06.295419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c56d9-b611-445a-9c13-9c4d69c5498c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:06.295690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c56d9-b611-445a-9c13-9c4d69c5498c admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:06.296329 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c56d9-b611-445a-9c13-9c4d69c5498c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:06.297972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c56d9-b611-445a-9c13-9c4d69c5498c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:06.307884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1205/2371] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:06.312016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9df2d-e4e0-43f9-8f8f-8e1d1ebf6eca admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:06.323118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9df2d-e4e0-43f9-8f8f-8e1d1ebf6eca admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:06.327788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04e06-9546-4b41-a31d-08df544a8091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:06.328469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04e06-9546-4b41-a31d-08df544a8091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:06.328751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04e06-9546-4b41-a31d-08df544a8091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:06.329021 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04e06-9546-4b41-a31d-08df544a8091 admin admin] PATH_INFO: `/v3/projects/4220bef6f7c5474b987bb3448a753754/users/9680b745737546e994bb51c8b6e2d6d4/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:06.330944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a04e06-9546-4b41-a31d-08df544a8091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4220bef6f7c5474b987bb3448a753754, user_id=9680b745737546e994bb51c8b6e2d6d4, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:06.334276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a04e06-9546-4b41-a31d-08df544a8091 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:06.345102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1167/2372] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:06 2022] PUT /identity/v3/projects/4220bef6f7c5474b987bb3448a753754/users/9680b745737546e994bb51c8b6e2d6d4/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:06.355555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0194e-7dc7-45da-8d82-e302cddb54a4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:06.355846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0194e-7dc7-45da-8d82-e302cddb54a4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:06.356172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0194e-7dc7-45da-8d82-e302cddb54a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:06.384209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7fc0194e-7dc7-45da-8d82-e302cddb54a4 None None] MFA Rules not processed for user `9680b745737546e994bb51c8b6e2d6d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:06.385512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7fc0194e-7dc7-45da-8d82-e302cddb54a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:06.465496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1206/2373] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:06.476231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04e06-9546-4b41-a31d-08df544a8091 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:06.477967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f3a04e06-9546-4b41-a31d-08df544a8091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:06.554017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04e06-9546-4b41-a31d-08df544a8091 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:06.557557 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c0828-eaf4-4ed0-a0c5-388d2af7e04a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:06.558156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c0828-eaf4-4ed0-a0c5-388d2af7e04a service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:06.558457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c0828-eaf4-4ed0-a0c5-388d2af7e04a service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:06.558728 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c0828-eaf4-4ed0-a0c5-388d2af7e04a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:06.568839 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c0828-eaf4-4ed0-a0c5-388d2af7e04a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:06.570275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c0828-eaf4-4ed0-a0c5-388d2af7e04a service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:06.579698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1168/2374] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:06.798156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb01d-ab87-412c-907e-f622fff5bd03 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:06.798442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb01d-ab87-412c-907e-f622fff5bd03 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:06.798715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb01d-ab87-412c-907e-f622fff5bd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:06.819786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f6ffb01d-ab87-412c-907e-f622fff5bd03 None None] MFA Rules not processed for user `a397314cdd52409c920071ae009516af`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:06.821134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f6ffb01d-ab87-412c-907e-f622fff5bd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:06.853051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:06.879195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1207/2375] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:07.106372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c0828-eaf4-4ed0-a0c5-388d2af7e04a service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:07.107813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e31c0828-eaf4-4ed0-a0c5-388d2af7e04a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:07.119515 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c0828-eaf4-4ed0-a0c5-388d2af7e04a service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:07.123059 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4feb3-302c-412d-879c-76f21d1765ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:07.123639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4feb3-302c-412d-879c-76f21d1765ef service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:07.123912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4feb3-302c-412d-879c-76f21d1765ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:07.124182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4feb3-302c-412d-879c-76f21d1765ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:07.186587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4feb3-302c-412d-879c-76f21d1765ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:07.188143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4feb3-302c-412d-879c-76f21d1765ef service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:07.197195 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1169/2376] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:11.265384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffb01d-ab87-412c-907e-f622fff5bd03 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:11.275280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffb01d-ab87-412c-907e-f622fff5bd03 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:11.278867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf61c41-2fd3-4f72-afee-fce0af451033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:11.279458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf61c41-2fd3-4f72-afee-fce0af451033 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:11.279733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf61c41-2fd3-4f72-afee-fce0af451033 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:11.280003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf61c41-2fd3-4f72-afee-fce0af451033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:11.335158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf61c41-2fd3-4f72-afee-fce0af451033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:11.336637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf61c41-2fd3-4f72-afee-fce0af451033 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:11.345394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1208/2377] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:13.561093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4feb3-302c-412d-879c-76f21d1765ef service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:13.571261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:13.631238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4feb3-302c-412d-879c-76f21d1765ef service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:13.636270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:13.636936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:13.637219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:13.637494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] PATH_INFO: `/v3/users/43d9e1179c9944ac8179c3e1ee66be6e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:13.638638 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d9e1179c9944ac8179c3e1ee66be6e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:13.640683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:13.690893 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d9e1179c9944ac8179c3e1ee66be6e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:13.699372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d9e1179c9944ac8179c3e1ee66be6e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:13.700250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d9e1179c9944ac8179c3e1ee66be6e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:13.710003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d9e1179c9944ac8179c3e1ee66be6e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:13.725115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d9e1179c9944ac8179c3e1ee66be6e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:13.733480 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1170/2378] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:13 2022] DELETE /identity/v3/users/43d9e1179c9944ac8179c3e1ee66be6e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:13.739468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57f6ca-c699-4cb2-90ee-c7c0ad281b6c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:13.739468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57f6ca-c699-4cb2-90ee-c7c0ad281b6c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:13.739468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57f6ca-c699-4cb2-90ee-c7c0ad281b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:13.758342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ec57f6ca-c699-4cb2-90ee-c7c0ad281b6c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:13.759560 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ec57f6ca-c699-4cb2-90ee-c7c0ad281b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:13.820704 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1209/2379] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:13.834658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:13.836475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:13.904994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3c4d7-5946-4fc9-9ffe-86b9b1da5adc admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:13.908806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf938cef-f36a-495d-91bf-d849dab1919b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:13.909492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf938cef-f36a-495d-91bf-d849dab1919b service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:13.910096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf938cef-f36a-495d-91bf-d849dab1919b service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:13.910448 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf938cef-f36a-495d-91bf-d849dab1919b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:13.920563 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf938cef-f36a-495d-91bf-d849dab1919b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:13.922094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf938cef-f36a-495d-91bf-d849dab1919b service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:13.932283 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1171/2380] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:14.645504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57f6ca-c699-4cb2-90ee-c7c0ad281b6c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:14.646956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ec57f6ca-c699-4cb2-90ee-c7c0ad281b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:14.658691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57f6ca-c699-4cb2-90ee-c7c0ad281b6c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:14.664091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:14.664776 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:14.665055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:14.665334 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] PATH_INFO: `/v3/projects/0a12aae8733e4adbba33c0fdd291b60f` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:14.666585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a12aae8733e4adbba33c0fdd291b60f)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:14.668361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:14.719879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a12aae8733e4adbba33c0fdd291b60f', 'request_id': 'req-ad10fa91-508e-4271-9316-32ee5a0a1aa3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:14.725977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a12aae8733e4adbba33c0fdd291b60f', 'request_id': 'req-ad10fa91-508e-4271-9316-32ee5a0a1aa3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:14.732641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a12aae8733e4adbba33c0fdd291b60f', 'request_id': 'req-ad10fa91-508e-4271-9316-32ee5a0a1aa3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:14.746023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] The token cache is being invalidate because project 0a12aae8733e4adbba33c0fdd291b60f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:14.746649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:14.747148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:14.748670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1210/2381] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:14 2022] DELETE /identity/v3/projects/0a12aae8733e4adbba33c0fdd291b60f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:14.752775 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf938cef-f36a-495d-91bf-d849dab1919b service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:14.764137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf938cef-f36a-495d-91bf-d849dab1919b service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:14.793721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:14.823525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:14.824158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:14.824467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:14.824759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] PATH_INFO: `/v3/users/2cac818f99de48c79c20186558e66f8a` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:14.825867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cac818f99de48c79c20186558e66f8a)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:14.827799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:14.876335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cac818f99de48c79c20186558e66f8a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:14.883547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cac818f99de48c79c20186558e66f8a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:14.884241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cac818f99de48c79c20186558e66f8a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:14.891572 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cac818f99de48c79c20186558e66f8a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:14.904347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cac818f99de48c79c20186558e66f8a'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:14.911110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1172/2382] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:14 2022] DELETE /identity/v3/users/2cac818f99de48c79c20186558e66f8a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:15.178830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:15.180501 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:15.189550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:15.194060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10fa91-508e-4271-9316-32ee5a0a1aa3 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:15.253795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:15.254429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:15.254706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:15.254978 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] PATH_INFO: `/v3/projects/426aa955e6154fdd89c6c6d3401b117e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:15.256635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=426aa955e6154fdd89c6c6d3401b117e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:15.258729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:15.303986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '426aa955e6154fdd89c6c6d3401b117e', 'request_id': 'req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.309399 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '426aa955e6154fdd89c6c6d3401b117e', 'request_id': 'req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.315275 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '426aa955e6154fdd89c6c6d3401b117e', 'request_id': 'req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.328212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] The token cache is being invalidate because project 426aa955e6154fdd89c6c6d3401b117e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:15.328531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.329143 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.330600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1211/2383] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:15 2022] DELETE /identity/v3/projects/426aa955e6154fdd89c6c6d3401b117e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:15.335366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:15.336798 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:15.349237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61b52d-6d65-42e1-b682-fb630272a204 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:15.405610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daac8a8-f31c-4d6f-80b4-2b58101b77d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:15.406245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daac8a8-f31c-4d6f-80b4-2b58101b77d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:15.406519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daac8a8-f31c-4d6f-80b4-2b58101b77d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:15.406787 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daac8a8-f31c-4d6f-80b4-2b58101b77d6 admin admin] PATH_INFO: `/v3/users/9680b745737546e994bb51c8b6e2d6d4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:15.407886 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daac8a8-f31c-4d6f-80b4-2b58101b77d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9680b745737546e994bb51c8b6e2d6d4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:15.410077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daac8a8-f31c-4d6f-80b4-2b58101b77d6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:15.455208 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3daac8a8-f31c-4d6f-80b4-2b58101b77d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9680b745737546e994bb51c8b6e2d6d4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.460193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:15.463718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3daac8a8-f31c-4d6f-80b4-2b58101b77d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9680b745737546e994bb51c8b6e2d6d4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.464485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3daac8a8-f31c-4d6f-80b4-2b58101b77d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9680b745737546e994bb51c8b6e2d6d4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.471715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3daac8a8-f31c-4d6f-80b4-2b58101b77d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9680b745737546e994bb51c8b6e2d6d4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.483868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3daac8a8-f31c-4d6f-80b4-2b58101b77d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9680b745737546e994bb51c8b6e2d6d4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.490081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1173/2384] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:15 2022] DELETE /identity/v3/users/9680b745737546e994bb51c8b6e2d6d4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:15.765623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:15.768395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:15.782582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854500f-d05f-4e78-a5e5-c7d7e7db5dd5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:15.850526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:15.851238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:15.851512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:15.851783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] PATH_INFO: `/v3/projects/4220bef6f7c5474b987bb3448a753754` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:15.853733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4220bef6f7c5474b987bb3448a753754)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:15.855465 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:15.897574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:15.908186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4220bef6f7c5474b987bb3448a753754', 'request_id': 'req-015e6382-d877-4ff5-828b-dd5585f4aa7b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.914715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4220bef6f7c5474b987bb3448a753754', 'request_id': 'req-015e6382-d877-4ff5-828b-dd5585f4aa7b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.921138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4220bef6f7c5474b987bb3448a753754', 'request_id': 'req-015e6382-d877-4ff5-828b-dd5585f4aa7b'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.934650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] The token cache is being invalidate because project 4220bef6f7c5474b987bb3448a753754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:15.934935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.935564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-015e6382-d877-4ff5-828b-dd5585f4aa7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:15.937096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1212/2385] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:15 2022] DELETE /identity/v3/projects/4220bef6f7c5474b987bb3448a753754 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:15.943138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ec53-b99c-425a-a1df-ee4d2bdc22ed None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:15.943619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ec53-b99c-425a-a1df-ee4d2bdc22ed None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:15.943726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98ec53-b99c-425a-a1df-ee4d2bdc22ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:15.963332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8f98ec53-b99c-425a-a1df-ee4d2bdc22ed None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:15.964635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8f98ec53-b99c-425a-a1df-ee4d2bdc22ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:16.025367 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1174/2386] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:37:16.033763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cece02-3bd1-4a8c-9030-81920a712257 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.033982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cece02-3bd1-4a8c-9030-81920a712257 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.034252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cece02-3bd1-4a8c-9030-81920a712257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.054949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-61cece02-3bd1-4a8c-9030-81920a712257 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:16.056139 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-61cece02-3bd1-4a8c-9030-81920a712257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:16.063895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1213/2387] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:16.068127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98ec53-b99c-425a-a1df-ee4d2bdc22ed None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.078265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98ec53-b99c-425a-a1df-ee4d2bdc22ed None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.083082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4334a6-3cb9-4d67-8119-213d5756f2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.083702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4334a6-3cb9-4d67-8119-213d5756f2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.083977 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4334a6-3cb9-4d67-8119-213d5756f2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.084250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4334a6-3cb9-4d67-8119-213d5756f2b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.084940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4334a6-3cb9-4d67-8119-213d5756f2b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:37:16.085235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4334a6-3cb9-4d67-8119-213d5756f2b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.086845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4334a6-3cb9-4d67-8119-213d5756f2b9 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:16.103713 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1175/2388] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:37:16.105509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cece02-3bd1-4a8c-9030-81920a712257 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.116472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cece02-3bd1-4a8c-9030-81920a712257 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.121309 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3b0c2-9a1e-426d-8a92-8e1fa051ea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.121939 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3b0c2-9a1e-426d-8a92-8e1fa051ea71 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.122218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3b0c2-9a1e-426d-8a92-8e1fa051ea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.122490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3b0c2-9a1e-426d-8a92-8e1fa051ea71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.123132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3b0c2-9a1e-426d-8a92-8e1fa051ea71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.125119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3b0c2-9a1e-426d-8a92-8e1fa051ea71 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:16.154664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1214/2389] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:37:16 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:16.159058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4334a6-3cb9-4d67-8119-213d5756f2b9 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.170694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4334a6-3cb9-4d67-8119-213d5756f2b9 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.234388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f784d-d930-44b2-a88e-a60d834b45d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.234992 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f784d-d930-44b2-a88e-a60d834b45d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.235265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f784d-d930-44b2-a88e-a60d834b45d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.235533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f784d-d930-44b2-a88e-a60d834b45d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.236232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f784d-d930-44b2-a88e-a60d834b45d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.238193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f784d-d930-44b2-a88e-a60d834b45d8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:16.260532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1176/2390] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:37:16 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:37:16.264765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3b0c2-9a1e-426d-8a92-8e1fa051ea71 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.275419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3b0c2-9a1e-426d-8a92-8e1fa051ea71 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.280009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e34dd-0933-4921-8aa6-c2f196a5d550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.280745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e34dd-0933-4921-8aa6-c2f196a5d550 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.281026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e34dd-0933-4921-8aa6-c2f196a5d550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.281300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e34dd-0933-4921-8aa6-c2f196a5d550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.281941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e34dd-0933-4921-8aa6-c2f196a5d550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.283560 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e34dd-0933-4921-8aa6-c2f196a5d550 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:16.294486 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1215/2391] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:16.298183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f784d-d930-44b2-a88e-a60d834b45d8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.308135 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f784d-d930-44b2-a88e-a60d834b45d8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.312658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f04ad2-bc8e-4ff4-b5a8-5106830f5af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.313237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04ad2-bc8e-4ff4-b5a8-5106830f5af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.313502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04ad2-bc8e-4ff4-b5a8-5106830f5af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.313766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04ad2-bc8e-4ff4-b5a8-5106830f5af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.314396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f04ad2-bc8e-4ff4-b5a8-5106830f5af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.316000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f04ad2-bc8e-4ff4-b5a8-5106830f5af4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:16.326358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1177/2392] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:16.334026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e34dd-0933-4921-8aa6-c2f196a5d550 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.343880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e34dd-0933-4921-8aa6-c2f196a5d550 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.348476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab300b9-0fd2-4812-a5ff-2ffc1d4e1c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.349699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab300b9-0fd2-4812-a5ff-2ffc1d4e1c47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.349972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab300b9-0fd2-4812-a5ff-2ffc1d4e1c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.350235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab300b9-0fd2-4812-a5ff-2ffc1d4e1c47 admin admin] PATH_INFO: `/v3/projects/872de989d79c487bad3719ac2e036451/users/9921072a139e4e6e9a2091f947f3f463/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.364463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab300b9-0fd2-4812-a5ff-2ffc1d4e1c47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=872de989d79c487bad3719ac2e036451, user_id=9921072a139e4e6e9a2091f947f3f463, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.367100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab300b9-0fd2-4812-a5ff-2ffc1d4e1c47 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:16.379702 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1216/2393] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:16 2022] PUT /identity/v3/projects/872de989d79c487bad3719ac2e036451/users/9921072a139e4e6e9a2091f947f3f463/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:16.391893 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da45cd-bf26-4f35-a6a6-5571dd303372 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.392179 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da45cd-bf26-4f35-a6a6-5571dd303372 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.392483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da45cd-bf26-4f35-a6a6-5571dd303372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.421648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d6da45cd-bf26-4f35-a6a6-5571dd303372 None None] MFA Rules not processed for user `9921072a139e4e6e9a2091f947f3f463`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:16.422821 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d6da45cd-bf26-4f35-a6a6-5571dd303372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:16.445838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:16.489398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1178/2394] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:16.493962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab300b9-0fd2-4812-a5ff-2ffc1d4e1c47 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.556004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab300b9-0fd2-4812-a5ff-2ffc1d4e1c47 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.560508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9384983-8cc0-49bc-9aed-e30729906e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.561114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384983-8cc0-49bc-9aed-e30729906e5e admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.561388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384983-8cc0-49bc-9aed-e30729906e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.561672 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384983-8cc0-49bc-9aed-e30729906e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.562326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9384983-8cc0-49bc-9aed-e30729906e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.564210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9384983-8cc0-49bc-9aed-e30729906e5e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:16.591736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1217/2395] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:37:16 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:16.595857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da45cd-bf26-4f35-a6a6-5571dd303372 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.606453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da45cd-bf26-4f35-a6a6-5571dd303372 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.662252 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac454cf-42a2-450a-9db3-eebe0f1afcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.662850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac454cf-42a2-450a-9db3-eebe0f1afcaa admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.663124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac454cf-42a2-450a-9db3-eebe0f1afcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.663393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac454cf-42a2-450a-9db3-eebe0f1afcaa admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.664016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac454cf-42a2-450a-9db3-eebe0f1afcaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.665962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac454cf-42a2-450a-9db3-eebe0f1afcaa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:16.686510 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1179/2396] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:37:16 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:37:16.690470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9384983-8cc0-49bc-9aed-e30729906e5e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.700995 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9384983-8cc0-49bc-9aed-e30729906e5e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.705537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e0dcc-8200-4898-bc4f-0dd3324eeeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.706125 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e0dcc-8200-4898-bc4f-0dd3324eeeef admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.706401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e0dcc-8200-4898-bc4f-0dd3324eeeef admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.706673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e0dcc-8200-4898-bc4f-0dd3324eeeef admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.707306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e0dcc-8200-4898-bc4f-0dd3324eeeef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.709472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e0dcc-8200-4898-bc4f-0dd3324eeeef admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:16.720635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1218/2397] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:16.724523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac454cf-42a2-450a-9db3-eebe0f1afcaa admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.734234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac454cf-42a2-450a-9db3-eebe0f1afcaa admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.738751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf31f1-344c-4f26-9df6-3971c8bec4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.739277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf31f1-344c-4f26-9df6-3971c8bec4bb admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.739549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf31f1-344c-4f26-9df6-3971c8bec4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.739816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf31f1-344c-4f26-9df6-3971c8bec4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.740476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf31f1-344c-4f26-9df6-3971c8bec4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.742084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf31f1-344c-4f26-9df6-3971c8bec4bb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:16.751804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1180/2398] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:16.756461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e0dcc-8200-4898-bc4f-0dd3324eeeef admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.766180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e0dcc-8200-4898-bc4f-0dd3324eeeef admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.770722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c37d71-f844-4e5f-a7ab-3a7d9c309483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.772647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c37d71-f844-4e5f-a7ab-3a7d9c309483 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.772934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c37d71-f844-4e5f-a7ab-3a7d9c309483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.773210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c37d71-f844-4e5f-a7ab-3a7d9c309483 admin admin] PATH_INFO: `/v3/projects/ff44319628214eaa959e07b6ec12bf12/users/e8dd1c0f556c480da025ed43e1ff5d88/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.787410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c37d71-f844-4e5f-a7ab-3a7d9c309483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff44319628214eaa959e07b6ec12bf12, user_id=e8dd1c0f556c480da025ed43e1ff5d88, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.790090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c37d71-f844-4e5f-a7ab-3a7d9c309483 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:16.807657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1219/2399] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:16 2022] PUT /identity/v3/projects/ff44319628214eaa959e07b6ec12bf12/users/e8dd1c0f556c480da025ed43e1ff5d88/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:16.818614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce2b01-83ae-4d1a-82cf-cc258c265d14 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.818892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce2b01-83ae-4d1a-82cf-cc258c265d14 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.819207 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce2b01-83ae-4d1a-82cf-cc258c265d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.846651 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-57ce2b01-83ae-4d1a-82cf-cc258c265d14 None None] MFA Rules not processed for user `e8dd1c0f556c480da025ed43e1ff5d88`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:16.847866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-57ce2b01-83ae-4d1a-82cf-cc258c265d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:16.909868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1181/2400] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:16.914428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c37d71-f844-4e5f-a7ab-3a7d9c309483 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:16.927361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:16.977479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c37d71-f844-4e5f-a7ab-3a7d9c309483 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:16.982163 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9fc97-9128-469c-b18b-c51aa62d992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:16.982770 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9fc97-9128-469c-b18b-c51aa62d992c admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:16.983046 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9fc97-9128-469c-b18b-c51aa62d992c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:16.983316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9fc97-9128-469c-b18b-c51aa62d992c admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:16.983949 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9fc97-9128-469c-b18b-c51aa62d992c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:16.986005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9fc97-9128-469c-b18b-c51aa62d992c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:17.013419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1220/2401] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:37:16 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:17.017668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce2b01-83ae-4d1a-82cf-cc258c265d14 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:17.028442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce2b01-83ae-4d1a-82cf-cc258c265d14 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:17.062517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:17.091037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69723543-1a3e-44cc-81df-1716ff777914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:17.091651 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69723543-1a3e-44cc-81df-1716ff777914 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:17.091926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69723543-1a3e-44cc-81df-1716ff777914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:17.092195 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69723543-1a3e-44cc-81df-1716ff777914 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:17.092843 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69723543-1a3e-44cc-81df-1716ff777914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:17.094788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69723543-1a3e-44cc-81df-1716ff777914 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:17.115710 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1182/2402] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:37:17 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:17.119759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9fc97-9128-469c-b18b-c51aa62d992c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:17.129869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9fc97-9128-469c-b18b-c51aa62d992c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:17.134535 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff19a2-e636-4f23-bf79-27654d1d8933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:17.135192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff19a2-e636-4f23-bf79-27654d1d8933 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:17.135475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff19a2-e636-4f23-bf79-27654d1d8933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:17.135746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff19a2-e636-4f23-bf79-27654d1d8933 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:17.136406 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff19a2-e636-4f23-bf79-27654d1d8933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:17.137987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff19a2-e636-4f23-bf79-27654d1d8933 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:17.150091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1221/2403] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:17.153861 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69723543-1a3e-44cc-81df-1716ff777914 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:17.164450 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69723543-1a3e-44cc-81df-1716ff777914 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:17.168947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecbe36-5d2e-4ea5-a9b9-cd81a239748a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:17.169577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecbe36-5d2e-4ea5-a9b9-cd81a239748a admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:17.169849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecbe36-5d2e-4ea5-a9b9-cd81a239748a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:17.170117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecbe36-5d2e-4ea5-a9b9-cd81a239748a admin admin] PATH_INFO: `/v3/projects/83c4b368c28f4fa79a254e4014291c8e/users/24ffb644ebbe489a952cbc26c9a263f7/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:17.185102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecbe36-5d2e-4ea5-a9b9-cd81a239748a admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c4b368c28f4fa79a254e4014291c8e, user_id=24ffb644ebbe489a952cbc26c9a263f7, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:17.187727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecbe36-5d2e-4ea5-a9b9-cd81a239748a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:17.199616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1183/2404] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:17 2022] PUT /identity/v3/projects/83c4b368c28f4fa79a254e4014291c8e/users/24ffb644ebbe489a952cbc26c9a263f7/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:17.203745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff19a2-e636-4f23-bf79-27654d1d8933 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:17.214072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff19a2-e636-4f23-bf79-27654d1d8933 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:17.270600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0779c23-1fbe-4166-aef7-1c44c043fa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:17.271757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0779c23-1fbe-4166-aef7-1c44c043fa56 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:17.272036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0779c23-1fbe-4166-aef7-1c44c043fa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:17.272304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0779c23-1fbe-4166-aef7-1c44c043fa56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:17.272967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0779c23-1fbe-4166-aef7-1c44c043fa56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:17.274550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0779c23-1fbe-4166-aef7-1c44c043fa56 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:17.283895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1222/2405] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:37:17.288880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecbe36-5d2e-4ea5-a9b9-cd81a239748a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:17.298773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecbe36-5d2e-4ea5-a9b9-cd81a239748a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:17.303293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bccec0-399b-4ee0-9b3a-e156b52741e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:17.303917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bccec0-399b-4ee0-9b3a-e156b52741e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:17.304189 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bccec0-399b-4ee0-9b3a-e156b52741e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:17.304491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bccec0-399b-4ee0-9b3a-e156b52741e1 admin admin] PATH_INFO: `/v3/projects/83c4b368c28f4fa79a254e4014291c8e/users/24ffb644ebbe489a952cbc26c9a263f7/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:17.306366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bccec0-399b-4ee0-9b3a-e156b52741e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c4b368c28f4fa79a254e4014291c8e, user_id=24ffb644ebbe489a952cbc26c9a263f7, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:17.309000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bccec0-399b-4ee0-9b3a-e156b52741e1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:17.319398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1184/2406] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:17 2022] PUT /identity/v3/projects/83c4b368c28f4fa79a254e4014291c8e/users/24ffb644ebbe489a952cbc26c9a263f7/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:17.384726 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb8f6-fbd2-4c1e-a6a4-c7462dd1ad0c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:17.384944 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb8f6-fbd2-4c1e-a6a4-c7462dd1ad0c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:17.385186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb8f6-fbd2-4c1e-a6a4-c7462dd1ad0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:17.418862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c6ecb8f6-fbd2-4c1e-a6a4-c7462dd1ad0c None None] MFA Rules not processed for user `24ffb644ebbe489a952cbc26c9a263f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:17.420063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c6ecb8f6-fbd2-4c1e-a6a4-c7462dd1ad0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:17.484850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1223/2407] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:17.496087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bccec0-399b-4ee0-9b3a-e156b52741e1 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:17.497794 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-27bccec0-399b-4ee0-9b3a-e156b52741e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:17.553635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bccec0-399b-4ee0-9b3a-e156b52741e1 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:17.557276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7abe0e-ba33-49a0-9929-fb0f91283744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:17.557864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7abe0e-ba33-49a0-9929-fb0f91283744 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:17.558142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7abe0e-ba33-49a0-9929-fb0f91283744 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:17.558411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7abe0e-ba33-49a0-9929-fb0f91283744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:17.567228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7abe0e-ba33-49a0-9929-fb0f91283744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:17.569179 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7abe0e-ba33-49a0-9929-fb0f91283744 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:17.578403 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1185/2408] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:17.618966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb8f6-fbd2-4c1e-a6a4-c7462dd1ad0c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:17.620562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c6ecb8f6-fbd2-4c1e-a6a4-c7462dd1ad0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:17.632992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb8f6-fbd2-4c1e-a6a4-c7462dd1ad0c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:17.636797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa20035-3a13-440e-bc05-5139d992653f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:17.637399 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa20035-3a13-440e-bc05-5139d992653f service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:17.637671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa20035-3a13-440e-bc05-5139d992653f service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:17.637933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa20035-3a13-440e-bc05-5139d992653f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:17.642099 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:17.694572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa20035-3a13-440e-bc05-5139d992653f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:17.695992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa20035-3a13-440e-bc05-5139d992653f service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:17.704788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1224/2409] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:20.603219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7abe0e-ba33-49a0-9929-fb0f91283744 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:20.669711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7abe0e-ba33-49a0-9929-fb0f91283744 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:20.674214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:20.674847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:20.675129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:20.675455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] PATH_INFO: `/v3/users/9921072a139e4e6e9a2091f947f3f463` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:20.676696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9921072a139e4e6e9a2091f947f3f463)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:20.678803 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:20.713511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:20.734350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9921072a139e4e6e9a2091f947f3f463'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:20.742556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9921072a139e4e6e9a2091f947f3f463'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:20.743272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9921072a139e4e6e9a2091f947f3f463'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:20.750797 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9921072a139e4e6e9a2091f947f3f463'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:20.763515 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9921072a139e4e6e9a2091f947f3f463'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:20.769512 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1186/2410] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:20 2022] DELETE /identity/v3/users/9921072a139e4e6e9a2091f947f3f463 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:20.774066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e8799-b38f-48ec-b17f-9b3e0cd84abb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:20.774563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e8799-b38f-48ec-b17f-9b3e0cd84abb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:20.774678 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e8799-b38f-48ec-b17f-9b3e0cd84abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:20.793584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b74e8799-b38f-48ec-b17f-9b3e0cd84abb None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:20.794792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b74e8799-b38f-48ec-b17f-9b3e0cd84abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:20.855880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1225/2411] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:20.866149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:20.867599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:20.926198 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f4597-4836-492f-bcde-53f0b89b05c4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:20.930285 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab952a88-d4c1-4cdb-a940-3d2452b2afad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:20.930886 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab952a88-d4c1-4cdb-a940-3d2452b2afad service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:20.931166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab952a88-d4c1-4cdb-a940-3d2452b2afad service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:20.931443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab952a88-d4c1-4cdb-a940-3d2452b2afad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:20.940446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab952a88-d4c1-4cdb-a940-3d2452b2afad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:20.941877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab952a88-d4c1-4cdb-a940-3d2452b2afad service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:20.952067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1187/2412] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:21.224793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e8799-b38f-48ec-b17f-9b3e0cd84abb None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:21.226293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b74e8799-b38f-48ec-b17f-9b3e0cd84abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:21.238391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e8799-b38f-48ec-b17f-9b3e0cd84abb None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:21.244283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:21.244772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:21.245046 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:21.245326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] PATH_INFO: `/v3/projects/872de989d79c487bad3719ac2e036451` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:21.246433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=872de989d79c487bad3719ac2e036451)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:21.248397 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:21.295500 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '872de989d79c487bad3719ac2e036451', 'request_id': 'req-6b31537f-84d8-4c40-8979-3f3f213b4348'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.301000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '872de989d79c487bad3719ac2e036451', 'request_id': 'req-6b31537f-84d8-4c40-8979-3f3f213b4348'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.307739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '872de989d79c487bad3719ac2e036451', 'request_id': 'req-6b31537f-84d8-4c40-8979-3f3f213b4348'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.320543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] The token cache is being invalidate because project 872de989d79c487bad3719ac2e036451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:21.321075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.321750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.323204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1226/2413] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:21 2022] DELETE /identity/v3/projects/872de989d79c487bad3719ac2e036451 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:37:21.327231 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab952a88-d4c1-4cdb-a940-3d2452b2afad service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:21.337125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab952a88-d4c1-4cdb-a940-3d2452b2afad service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:21.394288 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:21.395067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:21.395338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:21.395605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] PATH_INFO: `/v3/users/e8dd1c0f556c480da025ed43e1ff5d88` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:21.397952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8dd1c0f556c480da025ed43e1ff5d88)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:21.400158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:21.450485 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8dd1c0f556c480da025ed43e1ff5d88'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.455139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:21.460322 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8dd1c0f556c480da025ed43e1ff5d88'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.462729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8dd1c0f556c480da025ed43e1ff5d88'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.471416 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8dd1c0f556c480da025ed43e1ff5d88'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.483546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8dd1c0f556c480da025ed43e1ff5d88'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.489991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1188/2414] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:21 2022] DELETE /identity/v3/users/e8dd1c0f556c480da025ed43e1ff5d88 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:37:21.777378 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:21.780624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:21.796026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31537f-84d8-4c40-8979-3f3f213b4348 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:21.830266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:21.873985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:21.874659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:21.875451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:21.875730 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] PATH_INFO: `/v3/projects/ff44319628214eaa959e07b6ec12bf12` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:21.876880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff44319628214eaa959e07b6ec12bf12)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:21.878762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:21.933522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff44319628214eaa959e07b6ec12bf12', 'request_id': 'req-21101442-d0fe-417a-9e0f-e2e46dc23ad4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.939289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff44319628214eaa959e07b6ec12bf12', 'request_id': 'req-21101442-d0fe-417a-9e0f-e2e46dc23ad4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.946280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff44319628214eaa959e07b6ec12bf12', 'request_id': 'req-21101442-d0fe-417a-9e0f-e2e46dc23ad4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.961218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] The token cache is being invalidate because project ff44319628214eaa959e07b6ec12bf12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:21.961502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.962150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:21.963660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1227/2415] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:21 2022] DELETE /identity/v3/projects/ff44319628214eaa959e07b6ec12bf12 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:21.967767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:21.969218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:21.982852 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d095fb-f9da-49cc-ad7a-4ff7a18f451c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:22.053785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefddd6-ac52-4160-927a-f06128cf4927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:22.054776 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefddd6-ac52-4160-927a-f06128cf4927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:22.054776 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefddd6-ac52-4160-927a-f06128cf4927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:22.055203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefddd6-ac52-4160-927a-f06128cf4927 admin admin] PATH_INFO: `/v3/users/24ffb644ebbe489a952cbc26c9a263f7` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:22.056506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefddd6-ac52-4160-927a-f06128cf4927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24ffb644ebbe489a952cbc26c9a263f7)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:22.058545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefddd6-ac52-4160-927a-f06128cf4927 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:22.113825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6fefddd6-ac52-4160-927a-f06128cf4927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24ffb644ebbe489a952cbc26c9a263f7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:22.121924 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6fefddd6-ac52-4160-927a-f06128cf4927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24ffb644ebbe489a952cbc26c9a263f7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:22.123133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6fefddd6-ac52-4160-927a-f06128cf4927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24ffb644ebbe489a952cbc26c9a263f7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:22.132158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6fefddd6-ac52-4160-927a-f06128cf4927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24ffb644ebbe489a952cbc26c9a263f7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:22.148851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6fefddd6-ac52-4160-927a-f06128cf4927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24ffb644ebbe489a952cbc26c9a263f7'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:22.156445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1189/2416] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:21 2022] DELETE /identity/v3/users/24ffb644ebbe489a952cbc26c9a263f7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:22.445645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:22.447263 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:22.461316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21101442-d0fe-417a-9e0f-e2e46dc23ad4 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:22.517074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:22.517815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:22.518091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:22.518392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] PATH_INFO: `/v3/projects/83c4b368c28f4fa79a254e4014291c8e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:22.519545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=83c4b368c28f4fa79a254e4014291c8e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:22.521522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:22.572888 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83c4b368c28f4fa79a254e4014291c8e', 'request_id': 'req-b8939603-9b5c-4749-bfb5-8c31a3559b0e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:22.578554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83c4b368c28f4fa79a254e4014291c8e', 'request_id': 'req-b8939603-9b5c-4749-bfb5-8c31a3559b0e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:22.580469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:22.585987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83c4b368c28f4fa79a254e4014291c8e', 'request_id': 'req-b8939603-9b5c-4749-bfb5-8c31a3559b0e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:22.602214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] The token cache is being invalidate because project 83c4b368c28f4fa79a254e4014291c8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:22.602503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:22.603138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b8939603-9b5c-4749-bfb5-8c31a3559b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:22.604857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1228/2417] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:22 2022] DELETE /identity/v3/projects/83c4b368c28f4fa79a254e4014291c8e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:22.611394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089d35c-8fbb-473b-94fe-02e95be57f13 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:22.611766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089d35c-8fbb-473b-94fe-02e95be57f13 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:22.612067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089d35c-8fbb-473b-94fe-02e95be57f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:22.633691 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3089d35c-8fbb-473b-94fe-02e95be57f13 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:22.634958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3089d35c-8fbb-473b-94fe-02e95be57f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:22.686830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:22.702421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1190/2418] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:22.710506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ce1e8-2c81-46cd-9b22-5b28f61c70e7 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:22.710794 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ce1e8-2c81-46cd-9b22-5b28f61c70e7 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:22.711072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ce1e8-2c81-46cd-9b22-5b28f61c70e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:22.735180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f24ce1e8-2c81-46cd-9b22-5b28f61c70e7 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:22.736425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f24ce1e8-2c81-46cd-9b22-5b28f61c70e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:22.748553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1229/2419] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:37:22.749752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089d35c-8fbb-473b-94fe-02e95be57f13 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:22.760043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089d35c-8fbb-473b-94fe-02e95be57f13 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:22.764817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94409cdc-2f97-4fff-8d90-5d0c6482ce13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:22.765438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94409cdc-2f97-4fff-8d90-5d0c6482ce13 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:22.765716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94409cdc-2f97-4fff-8d90-5d0c6482ce13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:22.765986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94409cdc-2f97-4fff-8d90-5d0c6482ce13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:22.766794 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94409cdc-2f97-4fff-8d90-5d0c6482ce13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:37:22.767716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94409cdc-2f97-4fff-8d90-5d0c6482ce13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:22.769392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94409cdc-2f97-4fff-8d90-5d0c6482ce13 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:22.782308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1191/2420] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:37:22.787678 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ce1e8-2c81-46cd-9b22-5b28f61c70e7 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:22.799877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ce1e8-2c81-46cd-9b22-5b28f61c70e7 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:22.805932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bfb95-d762-4281-9f68-b445bb3e4011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:22.805932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bfb95-d762-4281-9f68-b445bb3e4011 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:22.806181 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bfb95-d762-4281-9f68-b445bb3e4011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:22.806389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bfb95-d762-4281-9f68-b445bb3e4011 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:22.806943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bfb95-d762-4281-9f68-b445bb3e4011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:22.809364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bfb95-d762-4281-9f68-b445bb3e4011 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:22.840404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1230/2421] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:37:22 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:37:22.842899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94409cdc-2f97-4fff-8d90-5d0c6482ce13 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:22.853428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94409cdc-2f97-4fff-8d90-5d0c6482ce13 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:22.913154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bbd5c-e0ba-4e7d-844d-478b84394dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:22.913754 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bbd5c-e0ba-4e7d-844d-478b84394dfb admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:22.914027 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bbd5c-e0ba-4e7d-844d-478b84394dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:22.914292 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bbd5c-e0ba-4e7d-844d-478b84394dfb admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:22.914914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709bbd5c-e0ba-4e7d-844d-478b84394dfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:22.916841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709bbd5c-e0ba-4e7d-844d-478b84394dfb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:22.939355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1192/2422] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:37:22 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:22.943841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bfb95-d762-4281-9f68-b445bb3e4011 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:22.954549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bfb95-d762-4281-9f68-b445bb3e4011 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:22.959736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2220f8-0c95-45b8-8b90-2cc10c021079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:22.960323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2220f8-0c95-45b8-8b90-2cc10c021079 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:22.960624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2220f8-0c95-45b8-8b90-2cc10c021079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:22.960887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2220f8-0c95-45b8-8b90-2cc10c021079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:22.961514 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2220f8-0c95-45b8-8b90-2cc10c021079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:22.963109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2220f8-0c95-45b8-8b90-2cc10c021079 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:22.973499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1231/2423] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:22.977659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bbd5c-e0ba-4e7d-844d-478b84394dfb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:22.987830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bbd5c-e0ba-4e7d-844d-478b84394dfb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:22.992990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cf097-6bc8-439d-bf4e-db23423456a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:22.993578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cf097-6bc8-439d-bf4e-db23423456a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:22.993855 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cf097-6bc8-439d-bf4e-db23423456a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:22.994155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cf097-6bc8-439d-bf4e-db23423456a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:22.994816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6cf097-6bc8-439d-bf4e-db23423456a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:22.996430 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6cf097-6bc8-439d-bf4e-db23423456a4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:23.007460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1193/2424] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:23.011514 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2220f8-0c95-45b8-8b90-2cc10c021079 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:23.021286 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2220f8-0c95-45b8-8b90-2cc10c021079 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:23.025781 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c88bd-9b08-49a1-84a1-f6086031fd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:23.026413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c88bd-9b08-49a1-84a1-f6086031fd8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:23.026686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c88bd-9b08-49a1-84a1-f6086031fd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:23.026952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c88bd-9b08-49a1-84a1-f6086031fd8b admin admin] PATH_INFO: `/v3/projects/a783a96596af4ad7b629431bae174c23/users/b481ee6298904947955f448284b6b132/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:23.041155 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c88bd-9b08-49a1-84a1-f6086031fd8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a783a96596af4ad7b629431bae174c23, user_id=b481ee6298904947955f448284b6b132, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:23.043745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c88bd-9b08-49a1-84a1-f6086031fd8b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:23.057068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1232/2425] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:23 2022] PUT /identity/v3/projects/a783a96596af4ad7b629431bae174c23/users/b481ee6298904947955f448284b6b132/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:23.069082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c100d9-5218-4923-a672-6087c8e022df None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:23.069365 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c100d9-5218-4923-a672-6087c8e022df None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:23.069632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c100d9-5218-4923-a672-6087c8e022df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:23.096252 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-00c100d9-5218-4923-a672-6087c8e022df None None] MFA Rules not processed for user `b481ee6298904947955f448284b6b132`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:23.097471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-00c100d9-5218-4923-a672-6087c8e022df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:23.160361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1194/2426] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:23.171512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c88bd-9b08-49a1-84a1-f6086031fd8b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:23.173240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-812c88bd-9b08-49a1-84a1-f6086031fd8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:23.229543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c88bd-9b08-49a1-84a1-f6086031fd8b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:23.233150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25514d3-6b05-479e-880c-4c71decacc37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:23.233743 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25514d3-6b05-479e-880c-4c71decacc37 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:23.234018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25514d3-6b05-479e-880c-4c71decacc37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:23.234288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25514d3-6b05-479e-880c-4c71decacc37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:23.242832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25514d3-6b05-479e-880c-4c71decacc37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:23.244192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25514d3-6b05-479e-880c-4c71decacc37 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:23.252739 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1233/2427] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:26.261932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c100d9-5218-4923-a672-6087c8e022df None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:26.307467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:26.324541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c100d9-5218-4923-a672-6087c8e022df None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:26.329401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:26.330028 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:26.330335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:26.330611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] PATH_INFO: `/v3/users/b481ee6298904947955f448284b6b132` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:26.331723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b481ee6298904947955f448284b6b132)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:26.333699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:26.379762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b481ee6298904947955f448284b6b132'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:26.386572 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b481ee6298904947955f448284b6b132'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:26.387261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b481ee6298904947955f448284b6b132'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:26.395046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b481ee6298904947955f448284b6b132'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:26.407544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b481ee6298904947955f448284b6b132'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:26.414094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1195/2428] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:26 2022] DELETE /identity/v3/users/b481ee6298904947955f448284b6b132 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:26.418573 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bedef7-fa38-4c76-be50-5f2e842b3927 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:26.419037 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bedef7-fa38-4c76-be50-5f2e842b3927 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:26.419131 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bedef7-fa38-4c76-be50-5f2e842b3927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:26.438019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-89bedef7-fa38-4c76-be50-5f2e842b3927 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:26.439208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-89bedef7-fa38-4c76-be50-5f2e842b3927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:26.497943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1234/2429] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:26.508045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:26.509510 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:26.586640 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d7f5b-61e4-42a0-a418-1cdde0fe9e36 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:26.590331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c9fb6-e12b-419b-b2e0-cc0f192263b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:26.590932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c9fb6-e12b-419b-b2e0-cc0f192263b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:26.591211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c9fb6-e12b-419b-b2e0-cc0f192263b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:26.591479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c9fb6-e12b-419b-b2e0-cc0f192263b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:26.601063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c9fb6-e12b-419b-b2e0-cc0f192263b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:26.602517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c9fb6-e12b-419b-b2e0-cc0f192263b1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:26.611727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1196/2430] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:26.884133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bedef7-fa38-4c76-be50-5f2e842b3927 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:26.885751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-89bedef7-fa38-4c76-be50-5f2e842b3927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:26.904062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bedef7-fa38-4c76-be50-5f2e842b3927 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:26.908884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:26.909515 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:26.910070 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:26.910391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] PATH_INFO: `/v3/projects/a783a96596af4ad7b629431bae174c23` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:26.911464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a783a96596af4ad7b629431bae174c23)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:26.913337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:26.938117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:26.960885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a783a96596af4ad7b629431bae174c23', 'request_id': 'req-180c6a41-0be2-461b-8b10-b9feb91151d3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:26.966423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a783a96596af4ad7b629431bae174c23', 'request_id': 'req-180c6a41-0be2-461b-8b10-b9feb91151d3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:26.972360 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a783a96596af4ad7b629431bae174c23', 'request_id': 'req-180c6a41-0be2-461b-8b10-b9feb91151d3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:26.985392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] The token cache is being invalidate because project a783a96596af4ad7b629431bae174c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:26.985675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:26.986295 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-180c6a41-0be2-461b-8b10-b9feb91151d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:26.987769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1235/2431] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:26 2022] DELETE /identity/v3/projects/a783a96596af4ad7b629431bae174c23 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:26.993317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eadc74-7ebe-4611-9978-167eae780b76 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:26.993701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eadc74-7ebe-4611-9978-167eae780b76 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:26.993985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eadc74-7ebe-4611-9978-167eae780b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:27.013748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-05eadc74-7ebe-4611-9978-167eae780b76 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:27.014874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-05eadc74-7ebe-4611-9978-167eae780b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:27.056882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:27.075126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1197/2432] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:27.084396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9e305-501b-46db-aaab-e3c9b9d9776e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:27.084709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9e305-501b-46db-aaab-e3c9b9d9776e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:27.084985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9e305-501b-46db-aaab-e3c9b9d9776e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:27.104712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-09a9e305-501b-46db-aaab-e3c9b9d9776e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:27.105913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-09a9e305-501b-46db-aaab-e3c9b9d9776e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:27.113787 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1236/2433] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:27.118457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eadc74-7ebe-4611-9978-167eae780b76 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:27.127936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eadc74-7ebe-4611-9978-167eae780b76 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:27.132617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066ac25-33f7-44d1-b1f6-d2c58760f837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:27.133204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066ac25-33f7-44d1-b1f6-d2c58760f837 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:27.133480 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066ac25-33f7-44d1-b1f6-d2c58760f837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:27.133751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066ac25-33f7-44d1-b1f6-d2c58760f837 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:27.134388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066ac25-33f7-44d1-b1f6-d2c58760f837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:37:27.134688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066ac25-33f7-44d1-b1f6-d2c58760f837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:27.136272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066ac25-33f7-44d1-b1f6-d2c58760f837 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:27.147777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1198/2434] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:37:27.151840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9e305-501b-46db-aaab-e3c9b9d9776e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:27.161001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9e305-501b-46db-aaab-e3c9b9d9776e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:27.166354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f8083-98b8-4541-8595-ef50d42a5717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:27.166948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f8083-98b8-4541-8595-ef50d42a5717 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:27.167227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f8083-98b8-4541-8595-ef50d42a5717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:27.167500 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f8083-98b8-4541-8595-ef50d42a5717 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:27.168152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f8083-98b8-4541-8595-ef50d42a5717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:27.170138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f8083-98b8-4541-8595-ef50d42a5717 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:27.199105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1237/2435] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:37:27 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:27.203420 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066ac25-33f7-44d1-b1f6-d2c58760f837 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:27.213158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066ac25-33f7-44d1-b1f6-d2c58760f837 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:27.269234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe792ec-9430-46f6-abae-9ed1dbb9aa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:27.269823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe792ec-9430-46f6-abae-9ed1dbb9aa49 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:27.270039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe792ec-9430-46f6-abae-9ed1dbb9aa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:27.270315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe792ec-9430-46f6-abae-9ed1dbb9aa49 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:27.270936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe792ec-9430-46f6-abae-9ed1dbb9aa49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:27.272904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe792ec-9430-46f6-abae-9ed1dbb9aa49 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:27.293900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1199/2436] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:37:27 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:27.298096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f8083-98b8-4541-8595-ef50d42a5717 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:27.308886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f8083-98b8-4541-8595-ef50d42a5717 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:27.313441 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cbde5-ca50-4ed5-8658-1a621f9f4e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:27.314031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cbde5-ca50-4ed5-8658-1a621f9f4e9b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:27.314308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cbde5-ca50-4ed5-8658-1a621f9f4e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:27.314583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cbde5-ca50-4ed5-8658-1a621f9f4e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:27.315244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90cbde5-ca50-4ed5-8658-1a621f9f4e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:27.316919 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90cbde5-ca50-4ed5-8658-1a621f9f4e9b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:27.326999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1238/2437] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:27.330662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe792ec-9430-46f6-abae-9ed1dbb9aa49 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:27.340585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe792ec-9430-46f6-abae-9ed1dbb9aa49 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:27.345118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c6c47-bd53-4a45-abd8-46dba11d9b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:27.345700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c6c47-bd53-4a45-abd8-46dba11d9b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:27.345971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c6c47-bd53-4a45-abd8-46dba11d9b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:27.346234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c6c47-bd53-4a45-abd8-46dba11d9b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:27.346888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c6c47-bd53-4a45-abd8-46dba11d9b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:27.348480 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c6c47-bd53-4a45-abd8-46dba11d9b55 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:27.358551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1200/2438] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:37:27.362599 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cbde5-ca50-4ed5-8658-1a621f9f4e9b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:27.373059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cbde5-ca50-4ed5-8658-1a621f9f4e9b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:27.377740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5c315-74ee-4559-a710-14e8b5ac6f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:27.378379 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5c315-74ee-4559-a710-14e8b5ac6f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:27.378657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5c315-74ee-4559-a710-14e8b5ac6f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:27.378930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5c315-74ee-4559-a710-14e8b5ac6f12 admin admin] PATH_INFO: `/v3/projects/8420d0e215ac4dbea6d49d22b3495fdd/users/ebd041e454684a3bb02fa0c429aacc23/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:27.393433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5c315-74ee-4559-a710-14e8b5ac6f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8420d0e215ac4dbea6d49d22b3495fdd, user_id=ebd041e454684a3bb02fa0c429aacc23, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:27.396117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5c315-74ee-4559-a710-14e8b5ac6f12 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:27.409508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1239/2439] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:27 2022] PUT /identity/v3/projects/8420d0e215ac4dbea6d49d22b3495fdd/users/ebd041e454684a3bb02fa0c429aacc23/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:27.420664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551b4c8-80a6-4952-aa37-87178cf603d7 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:27.420950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551b4c8-80a6-4952-aa37-87178cf603d7 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:27.421225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551b4c8-80a6-4952-aa37-87178cf603d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:27.448627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7551b4c8-80a6-4952-aa37-87178cf603d7 None None] MFA Rules not processed for user `ebd041e454684a3bb02fa0c429aacc23`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:27.449836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7551b4c8-80a6-4952-aa37-87178cf603d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:27.510875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1201/2440] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:27.521356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5c315-74ee-4559-a710-14e8b5ac6f12 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:27.523458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9df5c315-74ee-4559-a710-14e8b5ac6f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:27.579810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5c315-74ee-4559-a710-14e8b5ac6f12 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:27.583395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cc059-f069-4162-83f9-8415217638df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:27.583989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cc059-f069-4162-83f9-8415217638df service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:27.584265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cc059-f069-4162-83f9-8415217638df service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:27.585052 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cc059-f069-4162-83f9-8415217638df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:27.593190 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cc059-f069-4162-83f9-8415217638df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:27.594643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cc059-f069-4162-83f9-8415217638df service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:27.603998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1240/2441] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:37.508190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:37.599248 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=68,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:45.849025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551b4c8-80a6-4952-aa37-87178cf603d7 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:45.849025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7551b4c8-80a6-4952-aa37-87178cf603d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:45.938785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551b4c8-80a6-4952-aa37-87178cf603d7 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:45.949885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd8db1-2c40-4af8-82f7-f0cc4d246922 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:45.950506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd8db1-2c40-4af8-82f7-f0cc4d246922 service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:45.950784 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd8db1-2c40-4af8-82f7-f0cc4d246922 service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:45.951147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd8db1-2c40-4af8-82f7-f0cc4d246922 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:45.952940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b4fd8db1-2c40-4af8-82f7-f0cc4d246922 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:45.966104 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd8db1-2c40-4af8-82f7-f0cc4d246922 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:45.967681 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd8db1-2c40-4af8-82f7-f0cc4d246922 service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:45.977934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1202/2442] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:37:47.624019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cc059-f069-4162-83f9-8415217638df service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:47.625629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-673cc059-f069-4162-83f9-8415217638df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:47.683832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cc059-f069-4162-83f9-8415217638df service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:47.687414 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4211b6-895e-40be-bdcd-ffdb7731be07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:47.688013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4211b6-895e-40be-bdcd-ffdb7731be07 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:47.688291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4211b6-895e-40be-bdcd-ffdb7731be07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:47.688595 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4211b6-895e-40be-bdcd-ffdb7731be07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:47.690316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ab4211b6-895e-40be-bdcd-ffdb7731be07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:47.751452 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4211b6-895e-40be-bdcd-ffdb7731be07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:47.752923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4211b6-895e-40be-bdcd-ffdb7731be07 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:47.762223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1241/2443] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:50.811048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd8db1-2c40-4af8-82f7-f0cc4d246922 service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:50.826518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd8db1-2c40-4af8-82f7-f0cc4d246922 service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:50.833123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4911d-d900-4b36-a681-c2d400bdd5fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:50.833856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4911d-d900-4b36-a681-c2d400bdd5fc service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:50.834235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4911d-d900-4b36-a681-c2d400bdd5fc service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:50.834616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4911d-d900-4b36-a681-c2d400bdd5fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:50.837084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-20d4911d-d900-4b36-a681-c2d400bdd5fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:50.849615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4911d-d900-4b36-a681-c2d400bdd5fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:50.852205 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4911d-d900-4b36-a681-c2d400bdd5fc service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:50.864670 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1203/2444] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:50.940352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4211b6-895e-40be-bdcd-ffdb7731be07 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:50.950010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4211b6-895e-40be-bdcd-ffdb7731be07 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:50.955152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:50.955866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:50.956210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:50.956580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] PATH_INFO: `/v3/users/ebd041e454684a3bb02fa0c429aacc23` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:50.957754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebd041e454684a3bb02fa0c429aacc23)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:50.959731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:51.008631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebd041e454684a3bb02fa0c429aacc23'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:51.015653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebd041e454684a3bb02fa0c429aacc23'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:51.016974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebd041e454684a3bb02fa0c429aacc23'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:51.028849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebd041e454684a3bb02fa0c429aacc23'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:51.050686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebd041e454684a3bb02fa0c429aacc23'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:51.052021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1242/2445] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:50 2022] DELETE /identity/v3/users/ebd041e454684a3bb02fa0c429aacc23 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:51.057496 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d569008-6ba3-4cf9-a225-2b82d6288a35 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:51.057859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d569008-6ba3-4cf9-a225-2b82d6288a35 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:51.058202 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d569008-6ba3-4cf9-a225-2b82d6288a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:51.078767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-8d569008-6ba3-4cf9-a225-2b82d6288a35 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:51.079977 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8d569008-6ba3-4cf9-a225-2b82d6288a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:51.149265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1204/2446] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:51.158973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:51.160656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:51.190252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:51.247843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f56a5-8eaa-4b70-ad13-7d064cd89496 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:51.251764 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709bebf-ebfb-4e22-b256-59b5e332df8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:51.252616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709bebf-ebfb-4e22-b256-59b5e332df8c service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:51.253040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709bebf-ebfb-4e22-b256-59b5e332df8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:51.253363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709bebf-ebfb-4e22-b256-59b5e332df8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:51.266313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709bebf-ebfb-4e22-b256-59b5e332df8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:51.268003 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709bebf-ebfb-4e22-b256-59b5e332df8c service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:51.281260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1243/2447] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:37:51.561887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d569008-6ba3-4cf9-a225-2b82d6288a35 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:51.564071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8d569008-6ba3-4cf9-a225-2b82d6288a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:51.584851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d569008-6ba3-4cf9-a225-2b82d6288a35 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:51.589509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:51.590135 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:51.590415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:51.590688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] PATH_INFO: `/v3/projects/8420d0e215ac4dbea6d49d22b3495fdd` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:51.591803 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8420d0e215ac4dbea6d49d22b3495fdd)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:51.593794 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:51.642099 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8420d0e215ac4dbea6d49d22b3495fdd', 'request_id': 'req-81347323-dc33-46a6-a1ef-d29fb047497d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:51.643981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:51.648911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8420d0e215ac4dbea6d49d22b3495fdd', 'request_id': 'req-81347323-dc33-46a6-a1ef-d29fb047497d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:51.655974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8420d0e215ac4dbea6d49d22b3495fdd', 'request_id': 'req-81347323-dc33-46a6-a1ef-d29fb047497d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:51.669586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] The token cache is being invalidate because project 8420d0e215ac4dbea6d49d22b3495fdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:51.669874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:51.670519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:51.672019 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1205/2448] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:51 2022] DELETE /identity/v3/projects/8420d0e215ac4dbea6d49d22b3495fdd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:53.940478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709bebf-ebfb-4e22-b256-59b5e332df8c service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:53.942590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c709bebf-ebfb-4e22-b256-59b5e332df8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:53.957588 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709bebf-ebfb-4e22-b256-59b5e332df8c service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:54.014558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:54.015204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:54.015478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:54.015748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] PATH_INFO: `/v3/users/0e76ffd048d240ab94ee7e30b261d3cc` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:54.016987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e76ffd048d240ab94ee7e30b261d3cc)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:54.018929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:54.070857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e76ffd048d240ab94ee7e30b261d3cc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.077960 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e76ffd048d240ab94ee7e30b261d3cc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.078677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e76ffd048d240ab94ee7e30b261d3cc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.089485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e76ffd048d240ab94ee7e30b261d3cc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.102205 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e76ffd048d240ab94ee7e30b261d3cc'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.108101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1244/2449] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:53 2022] DELETE /identity/v3/users/0e76ffd048d240ab94ee7e30b261d3cc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:54.278021 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:54.279517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:54.297491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347323-dc33-46a6-a1ef-d29fb047497d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:54.357079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:54.357608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:54.357883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:54.358154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] PATH_INFO: `/v3/projects/4e940e1f7e3140f2865594bcdfc427a0` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:54.359261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e940e1f7e3140f2865594bcdfc427a0)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:54.361237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:54.411511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e940e1f7e3140f2865594bcdfc427a0', 'request_id': 'req-765ea782-091b-49da-805f-94e9d434be30'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.418005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e940e1f7e3140f2865594bcdfc427a0', 'request_id': 'req-765ea782-091b-49da-805f-94e9d434be30'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.424574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e940e1f7e3140f2865594bcdfc427a0', 'request_id': 'req-765ea782-091b-49da-805f-94e9d434be30'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.437504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] The token cache is being invalidate because project 4e940e1f7e3140f2865594bcdfc427a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:54.437850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.438469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.439964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1206/2450] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:54 2022] DELETE /identity/v3/projects/4e940e1f7e3140f2865594bcdfc427a0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:54.443932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:54.445616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:54.458645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028bac8-497a-48c9-aeab-d980644924f0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:54.500157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:54.516128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f690c7e-1882-4ceb-abfc-82497221b50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:54.516815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f690c7e-1882-4ceb-abfc-82497221b50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:54.517093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f690c7e-1882-4ceb-abfc-82497221b50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:54.517370 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f690c7e-1882-4ceb-abfc-82497221b50a admin admin] PATH_INFO: `/v3/users/21645f5c03d6474bb1fd5e8750b6b500` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:54.518496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f690c7e-1882-4ceb-abfc-82497221b50a admin admin] RBAC: Authorizing `identity:delete_user(user_id=21645f5c03d6474bb1fd5e8750b6b500)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:54.520585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f690c7e-1882-4ceb-abfc-82497221b50a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:54.567029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7f690c7e-1882-4ceb-abfc-82497221b50a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21645f5c03d6474bb1fd5e8750b6b500'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.574620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7f690c7e-1882-4ceb-abfc-82497221b50a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21645f5c03d6474bb1fd5e8750b6b500'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.575278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7f690c7e-1882-4ceb-abfc-82497221b50a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21645f5c03d6474bb1fd5e8750b6b500'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.582317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7f690c7e-1882-4ceb-abfc-82497221b50a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21645f5c03d6474bb1fd5e8750b6b500'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.593986 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7f690c7e-1882-4ceb-abfc-82497221b50a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21645f5c03d6474bb1fd5e8750b6b500'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.601284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1245/2451] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:37:54 2022] DELETE /identity/v3/users/21645f5c03d6474bb1fd5e8750b6b500 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:37:54.776286 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:54.778110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:54.794338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ea782-091b-49da-805f-94e9d434be30 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:54.851627 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:54.852263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:54.852573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:54.852982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] PATH_INFO: `/v3/projects/a780a162c99a4b82add018c78e7d4eb8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:54.854095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a780a162c99a4b82add018c78e7d4eb8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:54.856026 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:54.868238 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:54.909429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a780a162c99a4b82add018c78e7d4eb8', 'request_id': 'req-820a15f5-ad70-478b-98b9-db3b11f259dd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.915782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a780a162c99a4b82add018c78e7d4eb8', 'request_id': 'req-820a15f5-ad70-478b-98b9-db3b11f259dd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.924079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a780a162c99a4b82add018c78e7d4eb8', 'request_id': 'req-820a15f5-ad70-478b-98b9-db3b11f259dd'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.937848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] The token cache is being invalidate because project a780a162c99a4b82add018c78e7d4eb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:37:54.938132 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.938751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-820a15f5-ad70-478b-98b9-db3b11f259dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:37:54.940211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1207/2452] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:37:54 2022] DELETE /identity/v3/projects/a780a162c99a4b82add018c78e7d4eb8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:54.944905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6ab7e-faa8-4cdf-94ab-a3325c73626f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:54.945351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6ab7e-faa8-4cdf-94ab-a3325c73626f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:54.945449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6ab7e-faa8-4cdf-94ab-a3325c73626f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:54.965005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8df6ab7e-faa8-4cdf-94ab-a3325c73626f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:54.966210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8df6ab7e-faa8-4cdf-94ab-a3325c73626f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:55.026609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1246/2453] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:55.035927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5a2e-4716-4f2c-97cb-dd1eb88a9771 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:55.036212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5a2e-4716-4f2c-97cb-dd1eb88a9771 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:55.036541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5a2e-4716-4f2c-97cb-dd1eb88a9771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:55.066340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-773e5a2e-4716-4f2c-97cb-dd1eb88a9771 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:55.067625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-773e5a2e-4716-4f2c-97cb-dd1eb88a9771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:55.076141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1208/2454] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:55.080565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6ab7e-faa8-4cdf-94ab-a3325c73626f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:55.090100 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6ab7e-faa8-4cdf-94ab-a3325c73626f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:55.095213 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5df97-c1a0-42d5-ad92-2fe510dcd046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:55.095811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5df97-c1a0-42d5-ad92-2fe510dcd046 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:55.096093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5df97-c1a0-42d5-ad92-2fe510dcd046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:55.096375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5df97-c1a0-42d5-ad92-2fe510dcd046 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:55.097038 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5df97-c1a0-42d5-ad92-2fe510dcd046 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:37:55.097341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5df97-c1a0-42d5-ad92-2fe510dcd046 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:55.098928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5df97-c1a0-42d5-ad92-2fe510dcd046 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:55.111200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1247/2455] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:37:55.115631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e5a2e-4716-4f2c-97cb-dd1eb88a9771 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:55.125225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e5a2e-4716-4f2c-97cb-dd1eb88a9771 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:55.129823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2b34c-659a-447f-83cd-76132a68dca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:55.130411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2b34c-659a-447f-83cd-76132a68dca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:55.130683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2b34c-659a-447f-83cd-76132a68dca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:55.130945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2b34c-659a-447f-83cd-76132a68dca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:55.131570 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2b34c-659a-447f-83cd-76132a68dca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:55.133526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2b34c-659a-447f-83cd-76132a68dca5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:55.160491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1209/2456] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:37:55 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:55.165786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5df97-c1a0-42d5-ad92-2fe510dcd046 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:55.178842 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5df97-c1a0-42d5-ad92-2fe510dcd046 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:55.232005 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:37:55.243142 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f7569-0842-4ed1-97cc-c73ce83eac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:55.243753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f7569-0842-4ed1-97cc-c73ce83eac38 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:55.244271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f7569-0842-4ed1-97cc-c73ce83eac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:55.244856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f7569-0842-4ed1-97cc-c73ce83eac38 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:55.245604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f7569-0842-4ed1-97cc-c73ce83eac38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:55.248230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f7569-0842-4ed1-97cc-c73ce83eac38 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:55.273372 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1248/2457] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:37:55 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:37:55.277517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2b34c-659a-447f-83cd-76132a68dca5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:55.288900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2b34c-659a-447f-83cd-76132a68dca5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:55.293723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcbd3a-75bc-47b9-be55-468c4a98104c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:55.294323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcbd3a-75bc-47b9-be55-468c4a98104c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:55.294752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcbd3a-75bc-47b9-be55-468c4a98104c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:55.294868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcbd3a-75bc-47b9-be55-468c4a98104c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:55.295504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcbd3a-75bc-47b9-be55-468c4a98104c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:55.297120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcbd3a-75bc-47b9-be55-468c4a98104c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:55.307503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1210/2458] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:55.311578 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f7569-0842-4ed1-97cc-c73ce83eac38 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:55.321789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f7569-0842-4ed1-97cc-c73ce83eac38 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:55.326321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a584655-2959-4e7f-bac7-665159689fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:55.326905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584655-2959-4e7f-bac7-665159689fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:55.327176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584655-2959-4e7f-bac7-665159689fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:55.327443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584655-2959-4e7f-bac7-665159689fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:55.328094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a584655-2959-4e7f-bac7-665159689fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:55.329694 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a584655-2959-4e7f-bac7-665159689fd6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:55.340825 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1249/2459] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:37:55.344991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcbd3a-75bc-47b9-be55-468c4a98104c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:55.355433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcbd3a-75bc-47b9-be55-468c4a98104c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:55.360020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4edf8d-94f8-467d-a793-37427f5b628f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:55.360683 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4edf8d-94f8-467d-a793-37427f5b628f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:55.360960 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4edf8d-94f8-467d-a793-37427f5b628f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:55.361228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4edf8d-94f8-467d-a793-37427f5b628f admin admin] PATH_INFO: `/v3/projects/267f4c2e27aa4f9db62c3877d948f3d9/users/eccbf92faf2b4bad8746a2a48f5a6226/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:55.375275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4edf8d-94f8-467d-a793-37427f5b628f admin admin] RBAC: Authorizing `identity:create_grant(project_id=267f4c2e27aa4f9db62c3877d948f3d9, user_id=eccbf92faf2b4bad8746a2a48f5a6226, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:55.377914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4edf8d-94f8-467d-a793-37427f5b628f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:55.390722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1211/2460] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:37:55 2022] PUT /identity/v3/projects/267f4c2e27aa4f9db62c3877d948f3d9/users/eccbf92faf2b4bad8746a2a48f5a6226/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:37:55.395742 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62aa18-12f9-4625-9557-679fc25316dd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:55.396090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62aa18-12f9-4625-9557-679fc25316dd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:55.396555 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62aa18-12f9-4625-9557-679fc25316dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:55.416594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-cc62aa18-12f9-4625-9557-679fc25316dd None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:37:55.417707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-cc62aa18-12f9-4625-9557-679fc25316dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:55.474713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1250/2461] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:37:55.484939 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4edf8d-94f8-467d-a793-37427f5b628f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:37:55.486614 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ff4edf8d-94f8-467d-a793-37427f5b628f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:37:55.541119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4edf8d-94f8-467d-a793-37427f5b628f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:37:55.544673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c986e4-1755-4ac1-9cd0-e1820678f663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:37:55.545263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c986e4-1755-4ac1-9cd0-e1820678f663 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:37:55.545538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c986e4-1755-4ac1-9cd0-e1820678f663 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:37:55.545806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c986e4-1755-4ac1-9cd0-e1820678f663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:37:55.554435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c986e4-1755-4ac1-9cd0-e1820678f663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:37:55.555845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c986e4-1755-4ac1-9cd0-e1820678f663 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:37:55.564863 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1212/2462] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:38:00.232590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4923c-9eab-43a8-8b0a-4cbc95d08538 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:00.233928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4923c-9eab-43a8-8b0a-4cbc95d08538 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:00.234575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4923c-9eab-43a8-8b0a-4cbc95d08538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:00.269824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a3f4923c-9eab-43a8-8b0a-4cbc95d08538 None None] MFA Rules not processed for user `eccbf92faf2b4bad8746a2a48f5a6226`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:38:00.272929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a3f4923c-9eab-43a8-8b0a-4cbc95d08538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:38:00.359068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1251/2463] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:38:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:38:00.363921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c986e4-1755-4ac1-9cd0-e1820678f663 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:38:00.375618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c986e4-1755-4ac1-9cd0-e1820678f663 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:38:00.380936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0641c-1e21-4f5d-ae18-d94828336e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:38:00.381581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0641c-1e21-4f5d-ae18-d94828336e19 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:00.381864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0641c-1e21-4f5d-ae18-d94828336e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:00.382137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0641c-1e21-4f5d-ae18-d94828336e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:00.382828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0641c-1e21-4f5d-ae18-d94828336e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:38:00.384988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0641c-1e21-4f5d-ae18-d94828336e19 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:38:00.428029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1213/2464] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:38:00 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:38:00.432458 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4923c-9eab-43a8-8b0a-4cbc95d08538 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:38:00.449704 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4923c-9eab-43a8-8b0a-4cbc95d08538 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:38:00.518033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49dc9a-00a0-49fd-9092-cb677fe911de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:38:00.518821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49dc9a-00a0-49fd-9092-cb677fe911de admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:00.519242 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49dc9a-00a0-49fd-9092-cb677fe911de admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:00.519508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49dc9a-00a0-49fd-9092-cb677fe911de admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:00.520231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49dc9a-00a0-49fd-9092-cb677fe911de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:38:00.523099 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49dc9a-00a0-49fd-9092-cb677fe911de admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:38:00.532618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:38:00.552577 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1252/2465] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:38:00 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:38:00.555270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0641c-1e21-4f5d-ae18-d94828336e19 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:38:00.572765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0641c-1e21-4f5d-ae18-d94828336e19 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:38:00.578724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd2213-46ca-44a1-93e9-d3c880ddcc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:38:00.579790 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd2213-46ca-44a1-93e9-d3c880ddcc7a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:00.580077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd2213-46ca-44a1-93e9-d3c880ddcc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:00.580264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd2213-46ca-44a1-93e9-d3c880ddcc7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:00.581041 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd2213-46ca-44a1-93e9-d3c880ddcc7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:38:00.582649 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd2213-46ca-44a1-93e9-d3c880ddcc7a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:38:00.595411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1214/2466] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:38:00.599870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49dc9a-00a0-49fd-9092-cb677fe911de admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:38:00.611126 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49dc9a-00a0-49fd-9092-cb677fe911de admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:38:00.615813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ca30-bea1-4e83-aaaa-d4cc191499f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:38:00.616534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ca30-bea1-4e83-aaaa-d4cc191499f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:00.616810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ca30-bea1-4e83-aaaa-d4cc191499f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:00.617076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ca30-bea1-4e83-aaaa-d4cc191499f2 admin admin] PATH_INFO: `/v3/projects/1a1b2e39947444958f8dd829153d3a21/users/f70d52c9129c45e0b6b6e4a88c3d793c/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:00.635421 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103ca30-bea1-4e83-aaaa-d4cc191499f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a1b2e39947444958f8dd829153d3a21, user_id=f70d52c9129c45e0b6b6e4a88c3d793c, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:38:00.638663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103ca30-bea1-4e83-aaaa-d4cc191499f2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:38:00.661146 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1253/2467] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:38:00 2022] PUT /identity/v3/projects/1a1b2e39947444958f8dd829153d3a21/users/f70d52c9129c45e0b6b6e4a88c3d793c/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:38:00.662348 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd2213-46ca-44a1-93e9-d3c880ddcc7a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:38:00.675569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd2213-46ca-44a1-93e9-d3c880ddcc7a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:38:00.680832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:38:00.742217 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec94776-ab7b-417a-9e0b-ceb146752d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:38:00.742824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec94776-ab7b-417a-9e0b-ceb146752d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:00.743097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec94776-ab7b-417a-9e0b-ceb146752d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:00.743362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec94776-ab7b-417a-9e0b-ceb146752d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:00.743990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec94776-ab7b-417a-9e0b-ceb146752d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:38:00.745603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec94776-ab7b-417a-9e0b-ceb146752d95 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:38:00.755546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1215/2468] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:38:00.760576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ca30-bea1-4e83-aaaa-d4cc191499f2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:38:00.770620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ca30-bea1-4e83-aaaa-d4cc191499f2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:38:00.776285 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a404a-dcad-4d4d-bf0e-f3f004b129f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:38:00.777194 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a404a-dcad-4d4d-bf0e-f3f004b129f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:00.777468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a404a-dcad-4d4d-bf0e-f3f004b129f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:00.777767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a404a-dcad-4d4d-bf0e-f3f004b129f5 admin admin] PATH_INFO: `/v3/projects/1a1b2e39947444958f8dd829153d3a21/users/f70d52c9129c45e0b6b6e4a88c3d793c/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:00.779648 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a404a-dcad-4d4d-bf0e-f3f004b129f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a1b2e39947444958f8dd829153d3a21, user_id=f70d52c9129c45e0b6b6e4a88c3d793c, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:38:00.782385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a404a-dcad-4d4d-bf0e-f3f004b129f5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:38:00.793142 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1254/2469] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:38:00 2022] PUT /identity/v3/projects/1a1b2e39947444958f8dd829153d3a21/users/f70d52c9129c45e0b6b6e4a88c3d793c/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:38:01.733434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533f235-8a81-4b27-9b0d-6f42e6d87140 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:01.734587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533f235-8a81-4b27-9b0d-6f42e6d87140 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:01.735731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533f235-8a81-4b27-9b0d-6f42e6d87140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:01.767572 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-5533f235-8a81-4b27-9b0d-6f42e6d87140 None None] MFA Rules not processed for user `ccbdadc310fc4bf8a6679bcb39b34ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:38:01.769501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5533f235-8a81-4b27-9b0d-6f42e6d87140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:38:01.864620 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1216/2470] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:38:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:38:06.368065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a92b3-d9d4-4c56-98b8-7ae5dff2551b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:06.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a92b3-d9d4-4c56-98b8-7ae5dff2551b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:06.368703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a92b3-d9d4-4c56-98b8-7ae5dff2551b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:06.410377 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-945a92b3-d9d4-4c56-98b8-7ae5dff2551b None None] MFA Rules not processed for user `f70d52c9129c45e0b6b6e4a88c3d793c`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:38:06.412177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-945a92b3-d9d4-4c56-98b8-7ae5dff2551b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:38:06.532489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1255/2471] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:38:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:38:06.545766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533f235-8a81-4b27-9b0d-6f42e6d87140 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:38:06.548300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5533f235-8a81-4b27-9b0d-6f42e6d87140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:38:06.686196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533f235-8a81-4b27-9b0d-6f42e6d87140 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:38:06.695382 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b649b-fd09-432e-9e83-e9d9e36488e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:38:06.696068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b649b-fd09-432e-9e83-e9d9e36488e6 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:06.696353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b649b-fd09-432e-9e83-e9d9e36488e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:06.696679 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b649b-fd09-432e-9e83-e9d9e36488e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:06.717230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:38:06.762771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b649b-fd09-432e-9e83-e9d9e36488e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:38:06.764756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b649b-fd09-432e-9e83-e9d9e36488e6 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:38:06.775912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1217/2472] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:38:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:38:16.527112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:INSERT=5,keystone:SELECT=51 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:38:16.770741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=19 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:38:32.035556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2263185-2ad5-4bdf-add5-af18d27c28b8 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:32.036051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2263185-2ad5-4bdf-add5-af18d27c28b8 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:32.036211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2263185-2ad5-4bdf-add5-af18d27c28b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:32.054557 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-c2263185-2ad5-4bdf-add5-af18d27c28b8 None None] MFA Rules not processed for user `eccbf92faf2b4bad8746a2a48f5a6226`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:38:32.055742 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c2263185-2ad5-4bdf-add5-af18d27c28b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:38:32.062441 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1256/2473] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:38:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:38:42.049252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:38:46.009187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd6a9a-f710-4a59-a07e-a81a050d40dd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:46.009745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd6a9a-f710-4a59-a07e-a81a050d40dd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:46.009745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd6a9a-f710-4a59-a07e-a81a050d40dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:46.043159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b0bd6a9a-f710-4a59-a07e-a81a050d40dd None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:38:46.044922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b0bd6a9a-f710-4a59-a07e-a81a050d40dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:38:46.099851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1218/2474] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:38:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:38:46.147167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2263185-2ad5-4bdf-add5-af18d27c28b8 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:38:46.148658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c2263185-2ad5-4bdf-add5-af18d27c28b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:38:46.220813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2263185-2ad5-4bdf-add5-af18d27c28b8 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:38:46.225696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fc335-125b-420a-b965-e233dd58716c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:38:46.226304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fc335-125b-420a-b965-e233dd58716c service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:46.226569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fc335-125b-420a-b965-e233dd58716c service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:46.226835 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fc335-125b-420a-b965-e233dd58716c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:46.235428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fc335-125b-420a-b965-e233dd58716c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:38:46.237029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fc335-125b-420a-b965-e233dd58716c service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:38:46.246109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1257/2475] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:38:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:38:49.880380 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd6a9a-f710-4a59-a07e-a81a050d40dd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:38:49.943239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd6a9a-f710-4a59-a07e-a81a050d40dd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:38:49.946893 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc377a-8250-41b9-b274-54f915b5310f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:38:49.947565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc377a-8250-41b9-b274-54f915b5310f service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:38:49.947843 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc377a-8250-41b9-b274-54f915b5310f service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:38:49.948113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc377a-8250-41b9-b274-54f915b5310f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:38:49.958123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc377a-8250-41b9-b274-54f915b5310f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:38:49.959644 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc377a-8250-41b9-b274-54f915b5310f service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:38:49.972554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1219/2476] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:38:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:38:56.242818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=32 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:38:59.965943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=55 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:39:00.208139 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96c8f8-968e-4d5c-a7f1-9970fc57a9da None None] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:00.208670 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96c8f8-968e-4d5c-a7f1-9970fc57a9da None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:00.208798 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96c8f8-968e-4d5c-a7f1-9970fc57a9da None None] PATH_INFO: `/` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:00.210178 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1258/2477] 10.222.0.17 () {54 vars in 921 bytes} [Fri Sep 30 19:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 19:39:00.215419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b859f9c-efe9-4946-b6db-33cbbe16b39a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:00.216392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b859f9c-efe9-4946-b6db-33cbbe16b39a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:00.216705 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b859f9c-efe9-4946-b6db-33cbbe16b39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:00.236489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9b859f9c-efe9-4946-b6db-33cbbe16b39a None None] MFA Rules not processed for user `5952db29e6fc4ddbb9aeb0522497312c`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:39:00.237773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9b859f9c-efe9-4946-b6db-33cbbe16b39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:00.246883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1220/2478] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Sep 30 19:39:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:39:00.259095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96c8f8-968e-4d5c-a7f1-9970fc57a9da None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:00.262373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9e96c8f8-968e-4d5c-a7f1-9970fc57a9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:00.335109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96c8f8-968e-4d5c-a7f1-9970fc57a9da None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:00.339356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cde557-ca07-4c46-8fea-af5f1f41e68a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a9b371434243daa4176215daefb79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:00.339964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cde557-ca07-4c46-8fea-af5f1f41e68a service glance] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:00.340239 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cde557-ca07-4c46-8fea-af5f1f41e68a service glance] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:00.340598 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cde557-ca07-4c46-8fea-af5f1f41e68a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:00.360601 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cde557-ca07-4c46-8fea-af5f1f41e68a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:00.362449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cde557-ca07-4c46-8fea-af5f1f41e68a service glance] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:00.376127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1259/2479] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:39:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:39:01.422916 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b859f9c-efe9-4946-b6db-33cbbe16b39a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:01.432173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b859f9c-efe9-4946-b6db-33cbbe16b39a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:01.435730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46382927-91b1-4ac2-a555-645f928822b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:01.436313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46382927-91b1-4ac2-a555-645f928822b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:01.436619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46382927-91b1-4ac2-a555-645f928822b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:01.436888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46382927-91b1-4ac2-a555-645f928822b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:01.438580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-46382927-91b1-4ac2-a555-645f928822b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:01.453301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46382927-91b1-4ac2-a555-645f928822b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:01.454729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46382927-91b1-4ac2-a555-645f928822b5 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:01.465716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1221/2480] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:39:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:39:02.288777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c83b1-b38d-45c0-8b7f-2e4aed42211b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:02.289254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c83b1-b38d-45c0-8b7f-2e4aed42211b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:02.289365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c83b1-b38d-45c0-8b7f-2e4aed42211b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:02.307620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-108c83b1-b38d-45c0-8b7f-2e4aed42211b None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:39:02.308810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-108c83b1-b38d-45c0-8b7f-2e4aed42211b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:02.316131 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1260/2481] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:39:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:39:02.328307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46382927-91b1-4ac2-a555-645f928822b5 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:02.339401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46382927-91b1-4ac2-a555-645f928822b5 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:02.344214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e2bd2-aaa6-4636-80f0-4f73eeb17433 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:02.344830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e2bd2-aaa6-4636-80f0-4f73eeb17433 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:02.345103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e2bd2-aaa6-4636-80f0-4f73eeb17433 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:02.345372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e2bd2-aaa6-4636-80f0-4f73eeb17433 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:02.353037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e2bd2-aaa6-4636-80f0-4f73eeb17433 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:02.354592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e2bd2-aaa6-4636-80f0-4f73eeb17433 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:02.362365 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1222/2482] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:39:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:39:02.662080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c83b1-b38d-45c0-8b7f-2e4aed42211b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:02.663475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-108c83b1-b38d-45c0-8b7f-2e4aed42211b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:02.676266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c83b1-b38d-45c0-8b7f-2e4aed42211b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:02.680113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7c313-0013-4b84-8df9-ec875699d5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:02.680781 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c313-0013-4b84-8df9-ec875699d5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:02.681201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c313-0013-4b84-8df9-ec875699d5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:02.681477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7c313-0013-4b84-8df9-ec875699d5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:02.683304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-98c7c313-0013-4b84-8df9-ec875699d5f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:02.698258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c313-0013-4b84-8df9-ec875699d5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:02.699782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7c313-0013-4b84-8df9-ec875699d5f6 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:02.711196 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1261/2483] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:39:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:39:12.359211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=15 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:39:12.706108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=35 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:39:18.385411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e2bd2-aaa6-4636-80f0-4f73eeb17433 service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:18.397806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e2bd2-aaa6-4636-80f0-4f73eeb17433 service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:18.403321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f3e7a-8fc3-4d8c-a015-3a96022c3522 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:18.403716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f3e7a-8fc3-4d8c-a015-3a96022c3522 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:18.403989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f3e7a-8fc3-4d8c-a015-3a96022c3522 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:18.404255 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f3e7a-8fc3-4d8c-a015-3a96022c3522 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:18.406027 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8e4f3e7a-8fc3-4d8c-a015-3a96022c3522 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:18.483774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f3e7a-8fc3-4d8c-a015-3a96022c3522 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:18.485411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f3e7a-8fc3-4d8c-a015-3a96022c3522 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:18.503360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1223/2484] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:39:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:39:24.425157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e68584-61a4-4aaa-84d9-afad8d5a337a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:24.425590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e68584-61a4-4aaa-84d9-afad8d5a337a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:24.425709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e68584-61a4-4aaa-84d9-afad8d5a337a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:24.474830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-23e68584-61a4-4aaa-84d9-afad8d5a337a None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:39:24.476029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-23e68584-61a4-4aaa-84d9-afad8d5a337a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:24.492165 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1262/2485] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 30 19:39:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:39:24.504810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f3e7a-8fc3-4d8c-a015-3a96022c3522 service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:24.525900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f3e7a-8fc3-4d8c-a015-3a96022c3522 service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:24.530825 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b164e-69c9-4e7c-8f82-f469d0873293 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:24.531412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b164e-69c9-4e7c-8f82-f469d0873293 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:24.531691 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b164e-69c9-4e7c-8f82-f469d0873293 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:24.531965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b164e-69c9-4e7c-8f82-f469d0873293 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:24.540160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b164e-69c9-4e7c-8f82-f469d0873293 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:24.541767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b164e-69c9-4e7c-8f82-f469d0873293 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:24.550253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1224/2486] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:39:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:39:34.489264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=18 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:39:34.546490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=51 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:39:49.653813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e68584-61a4-4aaa-84d9-afad8d5a337a None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:49.663667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e68584-61a4-4aaa-84d9-afad8d5a337a None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:49.667265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fef7ee-3919-440d-b188-322343e6bb07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:49.667852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fef7ee-3919-440d-b188-322343e6bb07 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:49.668121 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fef7ee-3919-440d-b188-322343e6bb07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:49.668422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fef7ee-3919-440d-b188-322343e6bb07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:49.670120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a7fef7ee-3919-440d-b188-322343e6bb07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:49.857273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fef7ee-3919-440d-b188-322343e6bb07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:49.858696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fef7ee-3919-440d-b188-322343e6bb07 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:49.869108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1263/2487] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:39:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:39:52.737287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b164e-69c9-4e7c-8f82-f469d0873293 service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:52.738836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f14b164e-69c9-4e7c-8f82-f469d0873293 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:52.754391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b164e-69c9-4e7c-8f82-f469d0873293 service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:52.758970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a3b21-cee3-4c7e-8777-cd37ca570976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:52.759587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a3b21-cee3-4c7e-8777-cd37ca570976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:52.759866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a3b21-cee3-4c7e-8777-cd37ca570976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:52.760139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a3b21-cee3-4c7e-8777-cd37ca570976 admin admin] PATH_INFO: `/v3/users/a397314cdd52409c920071ae009516af` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:52.774351 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a3b21-cee3-4c7e-8777-cd37ca570976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a397314cdd52409c920071ae009516af)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:52.776313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a3b21-cee3-4c7e-8777-cd37ca570976 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:52.824813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-803a3b21-cee3-4c7e-8777-cd37ca570976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a397314cdd52409c920071ae009516af'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:39:52.832692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-803a3b21-cee3-4c7e-8777-cd37ca570976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a397314cdd52409c920071ae009516af'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:39:52.833456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-803a3b21-cee3-4c7e-8777-cd37ca570976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a397314cdd52409c920071ae009516af'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:39:52.841837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-803a3b21-cee3-4c7e-8777-cd37ca570976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a397314cdd52409c920071ae009516af'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:39:52.855026 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-803a3b21-cee3-4c7e-8777-cd37ca570976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a397314cdd52409c920071ae009516af'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:39:52.861271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1225/2488] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:39:52 2022] DELETE /identity/v3/users/a397314cdd52409c920071ae009516af => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:39:53.038826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fef7ee-3919-440d-b188-322343e6bb07 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:53.051087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fef7ee-3919-440d-b188-322343e6bb07 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:53.105971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:53.106594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:53.106868 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:53.107133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] PATH_INFO: `/v3/projects/ac5815ead619491680fb44af178e9cc0` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:53.117720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac5815ead619491680fb44af178e9cc0)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:53.119641 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:53.163729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac5815ead619491680fb44af178e9cc0', 'request_id': 'req-dd36a935-69f5-42a5-b3e7-65a29754d758'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:39:53.169077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac5815ead619491680fb44af178e9cc0', 'request_id': 'req-dd36a935-69f5-42a5-b3e7-65a29754d758'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:39:53.175824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac5815ead619491680fb44af178e9cc0', 'request_id': 'req-dd36a935-69f5-42a5-b3e7-65a29754d758'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:39:53.187361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] The token cache is being invalidate because project ac5815ead619491680fb44af178e9cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:39:53.187657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:39:53.189015 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-dd36a935-69f5-42a5-b3e7-65a29754d758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:39:53.190401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1264/2489] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:39:53 2022] DELETE /identity/v3/projects/ac5815ead619491680fb44af178e9cc0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:39:53.195460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710ca07-5eba-4b84-b011-6b1a40e5ffb5 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:53.195750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710ca07-5eba-4b84-b011-6b1a40e5ffb5 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:53.196030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710ca07-5eba-4b84-b011-6b1a40e5ffb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:53.215523 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1710ca07-5eba-4b84-b011-6b1a40e5ffb5 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:39:53.216740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1710ca07-5eba-4b84-b011-6b1a40e5ffb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:53.279112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1226/2490] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:39:53.287965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b66af-5b4f-4655-9033-cf4a74c9d78d None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:53.288189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b66af-5b4f-4655-9033-cf4a74c9d78d None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:53.288472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b66af-5b4f-4655-9033-cf4a74c9d78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:53.311725 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-249b66af-5b4f-4655-9033-cf4a74c9d78d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:39:53.313617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-249b66af-5b4f-4655-9033-cf4a74c9d78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:53.325040 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1265/2491] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:39:53.326218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710ca07-5eba-4b84-b011-6b1a40e5ffb5 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:53.337239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710ca07-5eba-4b84-b011-6b1a40e5ffb5 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:53.342314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666b431-1a65-410a-b92e-69777f19a2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:53.343011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666b431-1a65-410a-b92e-69777f19a2fe admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:53.343363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666b431-1a65-410a-b92e-69777f19a2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:53.343692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666b431-1a65-410a-b92e-69777f19a2fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:53.344447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666b431-1a65-410a-b92e-69777f19a2fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:39:53.344813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666b431-1a65-410a-b92e-69777f19a2fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:53.346511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666b431-1a65-410a-b92e-69777f19a2fe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:53.362058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1227/2492] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:39:53.377506 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b66af-5b4f-4655-9033-cf4a74c9d78d None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:53.389413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b66af-5b4f-4655-9033-cf4a74c9d78d None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:53.398015 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5e607-bf2c-403e-b6e5-bb3831dfdc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:53.398488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5e607-bf2c-403e-b6e5-bb3831dfdc06 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:53.399020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5e607-bf2c-403e-b6e5-bb3831dfdc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:53.399298 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5e607-bf2c-403e-b6e5-bb3831dfdc06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:53.400503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5e607-bf2c-403e-b6e5-bb3831dfdc06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:53.404478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5e607-bf2c-403e-b6e5-bb3831dfdc06 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:53.410906 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:39:53.445766 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1266/2493] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:39:53 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:39:53.450038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666b431-1a65-410a-b92e-69777f19a2fe admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:53.460031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666b431-1a65-410a-b92e-69777f19a2fe admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:53.503702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:39:53.516262 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da87105-3777-40f6-b6c5-b387c7b25198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:53.516894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da87105-3777-40f6-b6c5-b387c7b25198 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:53.517181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da87105-3777-40f6-b6c5-b387c7b25198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:53.517464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da87105-3777-40f6-b6c5-b387c7b25198 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:53.518092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da87105-3777-40f6-b6c5-b387c7b25198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:53.520034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da87105-3777-40f6-b6c5-b387c7b25198 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:53.540816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1228/2494] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:39:53 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:39:53.544855 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5e607-bf2c-403e-b6e5-bb3831dfdc06 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:53.555473 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5e607-bf2c-403e-b6e5-bb3831dfdc06 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:53.560029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac168d44-058d-4079-9150-a36832db3be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:53.560656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac168d44-058d-4079-9150-a36832db3be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:53.560934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac168d44-058d-4079-9150-a36832db3be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:53.561201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac168d44-058d-4079-9150-a36832db3be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:53.561833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac168d44-058d-4079-9150-a36832db3be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:53.563423 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac168d44-058d-4079-9150-a36832db3be0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:53.573019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1267/2495] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:39:53.576708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da87105-3777-40f6-b6c5-b387c7b25198 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:53.586416 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da87105-3777-40f6-b6c5-b387c7b25198 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:53.590847 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c67819-a1a5-49d3-95bd-b45873760de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:53.591425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c67819-a1a5-49d3-95bd-b45873760de3 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:53.591698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c67819-a1a5-49d3-95bd-b45873760de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:53.591962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c67819-a1a5-49d3-95bd-b45873760de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:53.592615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c67819-a1a5-49d3-95bd-b45873760de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:53.594697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c67819-a1a5-49d3-95bd-b45873760de3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:53.604942 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1229/2496] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:39:53.608862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac168d44-058d-4079-9150-a36832db3be0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:53.619276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac168d44-058d-4079-9150-a36832db3be0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:53.623748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471e7e0-7cde-4011-97f0-3516caca9b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:53.624408 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e7e0-7cde-4011-97f0-3516caca9b60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:53.624691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e7e0-7cde-4011-97f0-3516caca9b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:53.624961 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e7e0-7cde-4011-97f0-3516caca9b60 admin admin] PATH_INFO: `/v3/projects/c67f747be87147bc928d1653f41d02c4/users/71614a60cc2b4d039440a8313c0631a2/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:53.639212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471e7e0-7cde-4011-97f0-3516caca9b60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c67f747be87147bc928d1653f41d02c4, user_id=71614a60cc2b4d039440a8313c0631a2, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:53.641969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471e7e0-7cde-4011-97f0-3516caca9b60 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:53.653998 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1268/2497] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:39:53 2022] PUT /identity/v3/projects/c67f747be87147bc928d1653f41d02c4/users/71614a60cc2b4d039440a8313c0631a2/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:39:53.664873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1626b3-e2fd-4aa0-a984-2f5db0edb53f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:53.665156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1626b3-e2fd-4aa0-a984-2f5db0edb53f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:53.665420 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1626b3-e2fd-4aa0-a984-2f5db0edb53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:53.693400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-cb1626b3-e2fd-4aa0-a984-2f5db0edb53f None None] MFA Rules not processed for user `71614a60cc2b4d039440a8313c0631a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:39:53.694595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-cb1626b3-e2fd-4aa0-a984-2f5db0edb53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:53.758472 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1230/2498] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:39:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:39:53.763057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471e7e0-7cde-4011-97f0-3516caca9b60 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:53.822649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471e7e0-7cde-4011-97f0-3516caca9b60 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:53.827133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195cca2-3391-42a0-97c5-d6bd385313ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:53.827718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195cca2-3391-42a0-97c5-d6bd385313ef admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:53.827992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195cca2-3391-42a0-97c5-d6bd385313ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:53.828256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195cca2-3391-42a0-97c5-d6bd385313ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:53.829385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195cca2-3391-42a0-97c5-d6bd385313ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:53.831231 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195cca2-3391-42a0-97c5-d6bd385313ef admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:53.857702 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1269/2499] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:39:53 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:39:53.862903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1626b3-e2fd-4aa0-a984-2f5db0edb53f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:53.873534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1626b3-e2fd-4aa0-a984-2f5db0edb53f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:54.047672 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efa749-fc61-439e-889f-2ea22e343815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:54.048263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efa749-fc61-439e-889f-2ea22e343815 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:54.048560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efa749-fc61-439e-889f-2ea22e343815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:54.048831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efa749-fc61-439e-889f-2ea22e343815 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:54.049451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efa749-fc61-439e-889f-2ea22e343815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:54.051410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efa749-fc61-439e-889f-2ea22e343815 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:54.071628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1231/2500] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:39:53 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:39:54.075731 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195cca2-3391-42a0-97c5-d6bd385313ef admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:54.086095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195cca2-3391-42a0-97c5-d6bd385313ef admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:54.090482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e434a22-9cf0-45ea-94f6-643e3f909887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:54.091064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e434a22-9cf0-45ea-94f6-643e3f909887 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:54.091331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e434a22-9cf0-45ea-94f6-643e3f909887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:54.091593 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e434a22-9cf0-45ea-94f6-643e3f909887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:54.092212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e434a22-9cf0-45ea-94f6-643e3f909887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:54.093790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e434a22-9cf0-45ea-94f6-643e3f909887 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:54.103797 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1270/2501] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:39:54.107806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efa749-fc61-439e-889f-2ea22e343815 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:54.117457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efa749-fc61-439e-889f-2ea22e343815 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:54.122175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed699dd-0e59-4a78-8f8c-0e48acc8c102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:54.122965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed699dd-0e59-4a78-8f8c-0e48acc8c102 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:54.123307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed699dd-0e59-4a78-8f8c-0e48acc8c102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:54.123615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed699dd-0e59-4a78-8f8c-0e48acc8c102 admin admin] PATH_INFO: `/v3/projects/dd979d814b814c62914f6b7ce2660c90/users/30094401ba4145259d448b7b40b2477d/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:54.138480 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed699dd-0e59-4a78-8f8c-0e48acc8c102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd979d814b814c62914f6b7ce2660c90, user_id=30094401ba4145259d448b7b40b2477d, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:54.141779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed699dd-0e59-4a78-8f8c-0e48acc8c102 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:54.154301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1232/2502] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:39:54 2022] PUT /identity/v3/projects/dd979d814b814c62914f6b7ce2660c90/users/30094401ba4145259d448b7b40b2477d/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:39:54.158302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e434a22-9cf0-45ea-94f6-643e3f909887 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:54.168064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e434a22-9cf0-45ea-94f6-643e3f909887 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:54.221805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e42794-0b88-4a82-9018-85ed307fdacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:54.222393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e42794-0b88-4a82-9018-85ed307fdacd admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:54.222665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e42794-0b88-4a82-9018-85ed307fdacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:54.222925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e42794-0b88-4a82-9018-85ed307fdacd admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:54.223545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e42794-0b88-4a82-9018-85ed307fdacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:54.225151 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e42794-0b88-4a82-9018-85ed307fdacd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:54.234451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1271/2503] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:39:54.238564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed699dd-0e59-4a78-8f8c-0e48acc8c102 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:54.248768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed699dd-0e59-4a78-8f8c-0e48acc8c102 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:54.253940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb8a4c-4acf-4f07-9a2e-28accce8ebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:54.254576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb8a4c-4acf-4f07-9a2e-28accce8ebf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:54.254851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb8a4c-4acf-4f07-9a2e-28accce8ebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:54.255118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb8a4c-4acf-4f07-9a2e-28accce8ebf3 admin admin] PATH_INFO: `/v3/projects/dd979d814b814c62914f6b7ce2660c90/users/30094401ba4145259d448b7b40b2477d/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:54.257016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb8a4c-4acf-4f07-9a2e-28accce8ebf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd979d814b814c62914f6b7ce2660c90, user_id=30094401ba4145259d448b7b40b2477d, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:54.259611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb8a4c-4acf-4f07-9a2e-28accce8ebf3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:54.270081 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1233/2504] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:39:54 2022] PUT /identity/v3/projects/dd979d814b814c62914f6b7ce2660c90/users/30094401ba4145259d448b7b40b2477d/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:39:54.280255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb6e1-d55b-4229-ac25-ed62fe4b34bc None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:54.280569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb6e1-d55b-4229-ac25-ed62fe4b34bc None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:54.280845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb6e1-d55b-4229-ac25-ed62fe4b34bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:54.305884 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:39:54.308761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b30fb6e1-d55b-4229-ac25-ed62fe4b34bc None None] MFA Rules not processed for user `30094401ba4145259d448b7b40b2477d`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:39:54.310661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b30fb6e1-d55b-4229-ac25-ed62fe4b34bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:54.376256 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1272/2505] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:39:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:39:54.387492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb8a4c-4acf-4f07-9a2e-28accce8ebf3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:54.389260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5deb8a4c-4acf-4f07-9a2e-28accce8ebf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:39:54.404173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:39:54.470984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb8a4c-4acf-4f07-9a2e-28accce8ebf3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:54.474565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b3bfb-10e3-4215-aa1c-c95c4495819d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:54.475158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b3bfb-10e3-4215-aa1c-c95c4495819d service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:54.475433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b3bfb-10e3-4215-aa1c-c95c4495819d service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:54.475701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b3bfb-10e3-4215-aa1c-c95c4495819d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:54.484274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b3bfb-10e3-4215-aa1c-c95c4495819d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:54.485744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b3bfb-10e3-4215-aa1c-c95c4495819d service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:54.494760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1234/2506] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:39:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:39:54.569811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fb6e1-d55b-4229-ac25-ed62fe4b34bc None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:39:54.579045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fb6e1-d55b-4229-ac25-ed62fe4b34bc None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:39:54.582525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99895c5d-20f4-46c0-873d-efcb81c69ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:39:54.583106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99895c5d-20f4-46c0-873d-efcb81c69ced service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:39:54.583376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99895c5d-20f4-46c0-873d-efcb81c69ced service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:39:54.583637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99895c5d-20f4-46c0-873d-efcb81c69ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:39:54.637533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99895c5d-20f4-46c0-873d-efcb81c69ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:39:54.638861 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99895c5d-20f4-46c0-873d-efcb81c69ced service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:39:54.647643 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1273/2507] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:39:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:04.490731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=31 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:04.643658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=56 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:05.504453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b3bfb-10e3-4215-aa1c-c95c4495819d service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:05.569616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b3bfb-10e3-4215-aa1c-c95c4495819d service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:05.574184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:05.574852 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:05.575125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:05.575391 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] PATH_INFO: `/v3/users/71614a60cc2b4d039440a8313c0631a2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:05.576545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=71614a60cc2b4d039440a8313c0631a2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:05.578491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:05.639043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71614a60cc2b4d039440a8313c0631a2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:05.646752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71614a60cc2b4d039440a8313c0631a2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:05.647573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71614a60cc2b4d039440a8313c0631a2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:05.655995 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71614a60cc2b4d039440a8313c0631a2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:05.670711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71614a60cc2b4d039440a8313c0631a2'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:05.679542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1235/2508] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:05 2022] DELETE /identity/v3/users/71614a60cc2b4d039440a8313c0631a2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:05.684085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88ac03-b95b-4fe6-b1dd-424453a71216 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:05.684620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88ac03-b95b-4fe6-b1dd-424453a71216 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:05.684620 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88ac03-b95b-4fe6-b1dd-424453a71216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:05.706992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2a88ac03-b95b-4fe6-b1dd-424453a71216 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:05.708318 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2a88ac03-b95b-4fe6-b1dd-424453a71216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:05.783222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1274/2509] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:05.793093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:05.794473 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:05.859234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890209e-e268-427c-bd65-e8d95b237d5b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:05.863467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011972ef-0baf-43e1-be9b-d441595a44c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:05.864289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011972ef-0baf-43e1-be9b-d441595a44c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:05.864671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011972ef-0baf-43e1-be9b-d441595a44c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:05.865012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011972ef-0baf-43e1-be9b-d441595a44c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:05.875119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011972ef-0baf-43e1-be9b-d441595a44c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:05.876049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011972ef-0baf-43e1-be9b-d441595a44c5 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:05.886006 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1236/2510] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:06.218136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88ac03-b95b-4fe6-b1dd-424453a71216 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:06.220073 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2a88ac03-b95b-4fe6-b1dd-424453a71216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:06.232323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88ac03-b95b-4fe6-b1dd-424453a71216 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:06.237177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:06.237805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:06.238169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:06.238503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] PATH_INFO: `/v3/projects/c67f747be87147bc928d1653f41d02c4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:06.239701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c67f747be87147bc928d1653f41d02c4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:06.241700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:06.292046 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c67f747be87147bc928d1653f41d02c4', 'request_id': 'req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:06.297497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c67f747be87147bc928d1653f41d02c4', 'request_id': 'req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:06.303523 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c67f747be87147bc928d1653f41d02c4', 'request_id': 'req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:06.318413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] The token cache is being invalidate because project c67f747be87147bc928d1653f41d02c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:06.318812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:06.319332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:06.320878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1275/2511] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:06 2022] DELETE /identity/v3/projects/c67f747be87147bc928d1653f41d02c4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:06.324999 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011972ef-0baf-43e1-be9b-d441595a44c5 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:06.335030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011972ef-0baf-43e1-be9b-d441595a44c5 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:06.354039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:06.413024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9395e19-9eb2-4e40-ae9d-cc2eb218022a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:06.413761 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9395e19-9eb2-4e40-ae9d-cc2eb218022a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:06.414149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9395e19-9eb2-4e40-ae9d-cc2eb218022a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:06.414483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9395e19-9eb2-4e40-ae9d-cc2eb218022a admin admin] PATH_INFO: `/v3/users/30094401ba4145259d448b7b40b2477d` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:06.415730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9395e19-9eb2-4e40-ae9d-cc2eb218022a admin admin] RBAC: Authorizing `identity:delete_user(user_id=30094401ba4145259d448b7b40b2477d)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:06.417937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9395e19-9eb2-4e40-ae9d-cc2eb218022a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:06.476297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f9395e19-9eb2-4e40-ae9d-cc2eb218022a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30094401ba4145259d448b7b40b2477d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:06.484362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f9395e19-9eb2-4e40-ae9d-cc2eb218022a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30094401ba4145259d448b7b40b2477d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:06.485139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f9395e19-9eb2-4e40-ae9d-cc2eb218022a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30094401ba4145259d448b7b40b2477d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:06.501007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f9395e19-9eb2-4e40-ae9d-cc2eb218022a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30094401ba4145259d448b7b40b2477d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:06.520669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-f9395e19-9eb2-4e40-ae9d-cc2eb218022a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30094401ba4145259d448b7b40b2477d'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:06.530949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1237/2512] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:06 2022] DELETE /identity/v3/users/30094401ba4145259d448b7b40b2477d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:06.908400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:06.910792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:06.936120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2bf9c-5a40-4609-b03f-8a56c6c1bcd1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:06.999405 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:07.000171 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.000873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.001153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] PATH_INFO: `/v3/projects/dd979d814b814c62914f6b7ce2660c90` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.003113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd979d814b814c62914f6b7ce2660c90)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:07.005122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:07.040037 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:07.063792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd979d814b814c62914f6b7ce2660c90', 'request_id': 'req-b2e359f0-7183-4986-8d94-0dc23e9259ad'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:07.069859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd979d814b814c62914f6b7ce2660c90', 'request_id': 'req-b2e359f0-7183-4986-8d94-0dc23e9259ad'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:07.075971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd979d814b814c62914f6b7ce2660c90', 'request_id': 'req-b2e359f0-7183-4986-8d94-0dc23e9259ad'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:07.090370 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] The token cache is being invalidate because project dd979d814b814c62914f6b7ce2660c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:07.090708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:07.091340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-b2e359f0-7183-4986-8d94-0dc23e9259ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:07.093281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1276/2513] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:06 2022] DELETE /identity/v3/projects/dd979d814b814c62914f6b7ce2660c90 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:07.098583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5354be0-47b2-4d34-940c-e740ae21ad6a None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.098870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5354be0-47b2-4d34-940c-e740ae21ad6a None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.099285 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5354be0-47b2-4d34-940c-e740ae21ad6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.120903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-c5354be0-47b2-4d34-940c-e740ae21ad6a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:07.122194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c5354be0-47b2-4d34-940c-e740ae21ad6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:07.189388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1238/2514] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:07.198964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa1730-3955-4749-b5f3-d2edd69b9efd None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.199251 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa1730-3955-4749-b5f3-d2edd69b9efd None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.199571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa1730-3955-4749-b5f3-d2edd69b9efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.220745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a9aa1730-3955-4749-b5f3-d2edd69b9efd None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:07.221941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a9aa1730-3955-4749-b5f3-d2edd69b9efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:07.229950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1277/2515] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:07.234257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5354be0-47b2-4d34-940c-e740ae21ad6a None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:07.247962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5354be0-47b2-4d34-940c-e740ae21ad6a None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:07.252838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae1f76-d8b7-44c2-b431-22122125f86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:07.253442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae1f76-d8b7-44c2-b431-22122125f86c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.253718 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae1f76-d8b7-44c2-b431-22122125f86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.253986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae1f76-d8b7-44c2-b431-22122125f86c admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.254628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae1f76-d8b7-44c2-b431-22122125f86c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:07.254923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae1f76-d8b7-44c2-b431-22122125f86c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:07.256596 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae1f76-d8b7-44c2-b431-22122125f86c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:07.268852 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1239/2516] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:40:07.273216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa1730-3955-4749-b5f3-d2edd69b9efd None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:07.283291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa1730-3955-4749-b5f3-d2edd69b9efd None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:07.288170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1e62b-0b23-47ce-abb2-0684551ed0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:07.288808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1e62b-0b23-47ce-abb2-0684551ed0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.289225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1e62b-0b23-47ce-abb2-0684551ed0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.289442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1e62b-0b23-47ce-abb2-0684551ed0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.290930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1e62b-0b23-47ce-abb2-0684551ed0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:07.292859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1e62b-0b23-47ce-abb2-0684551ed0a1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:07.322475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1278/2517] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:07 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:07.326434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae1f76-d8b7-44c2-b431-22122125f86c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:07.337873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae1f76-d8b7-44c2-b431-22122125f86c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:07.367195 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:07.414312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d337e-7896-47b0-894c-850b52b59e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:07.415479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d337e-7896-47b0-894c-850b52b59e62 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.416083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d337e-7896-47b0-894c-850b52b59e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.416696 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d337e-7896-47b0-894c-850b52b59e62 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.417592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d337e-7896-47b0-894c-850b52b59e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:07.419884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d337e-7896-47b0-894c-850b52b59e62 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:07.447720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1240/2518] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:07 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:07.449044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1e62b-0b23-47ce-abb2-0684551ed0a1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:07.460065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1e62b-0b23-47ce-abb2-0684551ed0a1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:07.466808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fff9f0-4880-44bc-9677-ffb215b798d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:07.466808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fff9f0-4880-44bc-9677-ffb215b798d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.466808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fff9f0-4880-44bc-9677-ffb215b798d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.467188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fff9f0-4880-44bc-9677-ffb215b798d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.467627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fff9f0-4880-44bc-9677-ffb215b798d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:07.469888 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fff9f0-4880-44bc-9677-ffb215b798d8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:07.481283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1279/2519] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:07.485513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d337e-7896-47b0-894c-850b52b59e62 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:07.500572 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d337e-7896-47b0-894c-850b52b59e62 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:07.506607 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1af085-4af8-4835-bb1f-91cbd70ada4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:07.507258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1af085-4af8-4835-bb1f-91cbd70ada4e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.507535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1af085-4af8-4835-bb1f-91cbd70ada4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.507804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1af085-4af8-4835-bb1f-91cbd70ada4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.508490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1af085-4af8-4835-bb1f-91cbd70ada4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:07.510084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1af085-4af8-4835-bb1f-91cbd70ada4e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:07.522241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1241/2520] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:07.527081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fff9f0-4880-44bc-9677-ffb215b798d8 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:07.538122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fff9f0-4880-44bc-9677-ffb215b798d8 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:07.543675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd4054-988f-4b21-a2db-9cba097bd775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:07.544455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4054-988f-4b21-a2db-9cba097bd775 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.544676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4054-988f-4b21-a2db-9cba097bd775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.544985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4054-988f-4b21-a2db-9cba097bd775 admin admin] PATH_INFO: `/v3/projects/80bd4f0287074f4ea45bba7987f5d3a8/users/b30b05d44c164b838648ba89006e9a70/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.560656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd4054-988f-4b21-a2db-9cba097bd775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80bd4f0287074f4ea45bba7987f5d3a8, user_id=b30b05d44c164b838648ba89006e9a70, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:07.563362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd4054-988f-4b21-a2db-9cba097bd775 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:07.576958 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1280/2521] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:07 2022] PUT /identity/v3/projects/80bd4f0287074f4ea45bba7987f5d3a8/users/b30b05d44c164b838648ba89006e9a70/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:07.587697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf7414-f0fe-46d4-ab6d-0afc37f12f14 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.587975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf7414-f0fe-46d4-ab6d-0afc37f12f14 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.588236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf7414-f0fe-46d4-ab6d-0afc37f12f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.615887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dcbf7414-f0fe-46d4-ab6d-0afc37f12f14 None None] MFA Rules not processed for user `b30b05d44c164b838648ba89006e9a70`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:07.617122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dcbf7414-f0fe-46d4-ab6d-0afc37f12f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:07.678994 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1242/2522] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:07.683504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd4054-988f-4b21-a2db-9cba097bd775 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:07.747107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd4054-988f-4b21-a2db-9cba097bd775 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:07.753889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43fd93-aa99-4bb9-a714-b7400db243ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:07.754492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43fd93-aa99-4bb9-a714-b7400db243ee admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.754769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43fd93-aa99-4bb9-a714-b7400db243ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.755038 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43fd93-aa99-4bb9-a714-b7400db243ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.755659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43fd93-aa99-4bb9-a714-b7400db243ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:07.757568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43fd93-aa99-4bb9-a714-b7400db243ee admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:07.784341 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1281/2523] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:07 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:07.788583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf7414-f0fe-46d4-ab6d-0afc37f12f14 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:07.799007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf7414-f0fe-46d4-ab6d-0afc37f12f14 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:07.852901 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98de44-abea-4103-89a1-bd7ca6f9a277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:07.853428 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98de44-abea-4103-89a1-bd7ca6f9a277 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.853702 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98de44-abea-4103-89a1-bd7ca6f9a277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.853967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98de44-abea-4103-89a1-bd7ca6f9a277 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.854593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98de44-abea-4103-89a1-bd7ca6f9a277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:07.856501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98de44-abea-4103-89a1-bd7ca6f9a277 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:07.876989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1243/2524] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:07 2022] POST /identity/v3/users => generated 300 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:40:07.881074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43fd93-aa99-4bb9-a714-b7400db243ee admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:07.891755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43fd93-aa99-4bb9-a714-b7400db243ee admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:07.896356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cf936-baee-44ca-949c-16b240aa2202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:07.896967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf936-baee-44ca-949c-16b240aa2202 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.897239 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf936-baee-44ca-949c-16b240aa2202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.897505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf936-baee-44ca-949c-16b240aa2202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.898134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cf936-baee-44ca-949c-16b240aa2202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:07.899693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cf936-baee-44ca-949c-16b240aa2202 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:07.910938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1282/2525] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:07.915188 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98de44-abea-4103-89a1-bd7ca6f9a277 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:07.925390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98de44-abea-4103-89a1-bd7ca6f9a277 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:07.929965 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954ef42-098c-4c91-89f5-b0f33bc7d78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:07.930603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954ef42-098c-4c91-89f5-b0f33bc7d78f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:07.930901 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954ef42-098c-4c91-89f5-b0f33bc7d78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:07.931151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954ef42-098c-4c91-89f5-b0f33bc7d78f admin admin] PATH_INFO: `/v3/projects/faf0063a11384b9d93d66ad6d378dd2c/users/8890e57be9b14f83b7fa1e50ce0edbe1/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:07.946214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954ef42-098c-4c91-89f5-b0f33bc7d78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf0063a11384b9d93d66ad6d378dd2c, user_id=8890e57be9b14f83b7fa1e50ce0edbe1, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:07.948873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954ef42-098c-4c91-89f5-b0f33bc7d78f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:07.961399 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1244/2526] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:07 2022] PUT /identity/v3/projects/faf0063a11384b9d93d66ad6d378dd2c/users/8890e57be9b14f83b7fa1e50ce0edbe1/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:07.965407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cf936-baee-44ca-949c-16b240aa2202 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:07.975037 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cf936-baee-44ca-949c-16b240aa2202 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:07.993824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:08.030795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6207a-837a-4af5-8475-f0de968274c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:08.031396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6207a-837a-4af5-8475-f0de968274c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:08.031672 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6207a-837a-4af5-8475-f0de968274c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:08.031942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6207a-837a-4af5-8475-f0de968274c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:08.032602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6207a-837a-4af5-8475-f0de968274c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:08.034172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6207a-837a-4af5-8475-f0de968274c3 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:08.044067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1283/2527] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:40:08.048800 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954ef42-098c-4c91-89f5-b0f33bc7d78f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:08.054134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:08.060539 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954ef42-098c-4c91-89f5-b0f33bc7d78f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:08.066155 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76651b-bb3e-411b-a937-8e248e073a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:08.066795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76651b-bb3e-411b-a937-8e248e073a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:08.067093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76651b-bb3e-411b-a937-8e248e073a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:08.067343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76651b-bb3e-411b-a937-8e248e073a34 admin admin] PATH_INFO: `/v3/projects/faf0063a11384b9d93d66ad6d378dd2c/users/8890e57be9b14f83b7fa1e50ce0edbe1/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:08.069366 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76651b-bb3e-411b-a937-8e248e073a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf0063a11384b9d93d66ad6d378dd2c, user_id=8890e57be9b14f83b7fa1e50ce0edbe1, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:08.072020 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76651b-bb3e-411b-a937-8e248e073a34 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:08.081985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1245/2528] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:08 2022] PUT /identity/v3/projects/faf0063a11384b9d93d66ad6d378dd2c/users/8890e57be9b14f83b7fa1e50ce0edbe1/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:08.092917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b289e4-55c1-46e4-8cb7-2fc6c12c31b3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:08.093198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b289e4-55c1-46e4-8cb7-2fc6c12c31b3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:08.093463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b289e4-55c1-46e4-8cb7-2fc6c12c31b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:08.120321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-87b289e4-55c1-46e4-8cb7-2fc6c12c31b3 None None] MFA Rules not processed for user `8890e57be9b14f83b7fa1e50ce0edbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:08.121462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-87b289e4-55c1-46e4-8cb7-2fc6c12c31b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:08.188498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1284/2529] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:08.205314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76651b-bb3e-411b-a937-8e248e073a34 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:08.206996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4b76651b-bb3e-411b-a937-8e248e073a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:08.266390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76651b-bb3e-411b-a937-8e248e073a34 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:08.270012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c85bcd-0591-4c8e-b778-f25846ac050e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:08.270613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c85bcd-0591-4c8e-b778-f25846ac050e service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:08.270891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c85bcd-0591-4c8e-b778-f25846ac050e service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:08.271157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c85bcd-0591-4c8e-b778-f25846ac050e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:08.280136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c85bcd-0591-4c8e-b778-f25846ac050e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:08.281628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c85bcd-0591-4c8e-b778-f25846ac050e service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:08.291922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1246/2530] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:08.897144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b289e4-55c1-46e4-8cb7-2fc6c12c31b3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:08.957609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b289e4-55c1-46e4-8cb7-2fc6c12c31b3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:08.962224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:08.962850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:08.963125 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:08.963393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] PATH_INFO: `/v3/users/b30b05d44c164b838648ba89006e9a70` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:08.964540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30b05d44c164b838648ba89006e9a70)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:08.966428 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:09.011999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30b05d44c164b838648ba89006e9a70'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.013189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:09.020307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30b05d44c164b838648ba89006e9a70'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.021353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30b05d44c164b838648ba89006e9a70'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.033389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30b05d44c164b838648ba89006e9a70'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.045844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30b05d44c164b838648ba89006e9a70'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.052484 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1285/2531] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:08 2022] DELETE /identity/v3/users/b30b05d44c164b838648ba89006e9a70 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:09.056997 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009da9-2e68-4b5f-8352-3e7ffd2ccd57 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:09.057588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009da9-2e68-4b5f-8352-3e7ffd2ccd57 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:09.057588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90009da9-2e68-4b5f-8352-3e7ffd2ccd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:09.076902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-90009da9-2e68-4b5f-8352-3e7ffd2ccd57 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:09.078091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-90009da9-2e68-4b5f-8352-3e7ffd2ccd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:09.139513 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1247/2532] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:09.149335 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:09.150841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:09.214089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf3746-6a4b-452b-8bc8-ed7cd349848b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:09.219942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcd271-17af-4052-ae88-be8f0a6d3dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:09.220509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcd271-17af-4052-ae88-be8f0a6d3dea service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:09.220788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcd271-17af-4052-ae88-be8f0a6d3dea service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:09.221233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcd271-17af-4052-ae88-be8f0a6d3dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:09.231575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcd271-17af-4052-ae88-be8f0a6d3dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:09.233158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcd271-17af-4052-ae88-be8f0a6d3dea service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:09.243689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1286/2533] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:09.500024 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90009da9-2e68-4b5f-8352-3e7ffd2ccd57 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:09.501479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-90009da9-2e68-4b5f-8352-3e7ffd2ccd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:09.514466 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90009da9-2e68-4b5f-8352-3e7ffd2ccd57 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:09.520350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:09.521056 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:09.521332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:09.521598 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] PATH_INFO: `/v3/projects/80bd4f0287074f4ea45bba7987f5d3a8` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:09.522716 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=80bd4f0287074f4ea45bba7987f5d3a8)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:09.524707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:09.572908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80bd4f0287074f4ea45bba7987f5d3a8', 'request_id': 'req-b477e298-af2c-44b4-885f-4ec616809b1e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.578772 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80bd4f0287074f4ea45bba7987f5d3a8', 'request_id': 'req-b477e298-af2c-44b4-885f-4ec616809b1e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.584885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80bd4f0287074f4ea45bba7987f5d3a8', 'request_id': 'req-b477e298-af2c-44b4-885f-4ec616809b1e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.597568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] The token cache is being invalidate because project 80bd4f0287074f4ea45bba7987f5d3a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:09.597851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.598455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.599941 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1248/2534] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:09 2022] DELETE /identity/v3/projects/80bd4f0287074f4ea45bba7987f5d3a8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:09.603876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcd271-17af-4052-ae88-be8f0a6d3dea service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:09.613217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcd271-17af-4052-ae88-be8f0a6d3dea service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:09.667269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c182a-5a7c-46d7-b206-503f305a63eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:09.667899 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c182a-5a7c-46d7-b206-503f305a63eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:09.668158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c182a-5a7c-46d7-b206-503f305a63eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:09.668516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c182a-5a7c-46d7-b206-503f305a63eb admin admin] PATH_INFO: `/v3/users/8890e57be9b14f83b7fa1e50ce0edbe1` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:09.669614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c182a-5a7c-46d7-b206-503f305a63eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8890e57be9b14f83b7fa1e50ce0edbe1)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:09.671501 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c182a-5a7c-46d7-b206-503f305a63eb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:09.714517 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-bf9c182a-5a7c-46d7-b206-503f305a63eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8890e57be9b14f83b7fa1e50ce0edbe1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.722096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-bf9c182a-5a7c-46d7-b206-503f305a63eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8890e57be9b14f83b7fa1e50ce0edbe1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.722825 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-bf9c182a-5a7c-46d7-b206-503f305a63eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8890e57be9b14f83b7fa1e50ce0edbe1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.731582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-bf9c182a-5a7c-46d7-b206-503f305a63eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8890e57be9b14f83b7fa1e50ce0edbe1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.736744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:09.745048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-bf9c182a-5a7c-46d7-b206-503f305a63eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8890e57be9b14f83b7fa1e50ce0edbe1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:09.750789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1287/2535] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:09 2022] DELETE /identity/v3/users/8890e57be9b14f83b7fa1e50ce0edbe1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:10.020865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:10.022350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:10.037339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477e298-af2c-44b4-885f-4ec616809b1e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:10.050117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:10.097178 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:10.097801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.098070 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.098337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] PATH_INFO: `/v3/projects/faf0063a11384b9d93d66ad6d378dd2c` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.099432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faf0063a11384b9d93d66ad6d378dd2c)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:10.101369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:10.146129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faf0063a11384b9d93d66ad6d378dd2c', 'request_id': 'req-5f0964cc-c8c1-4c45-9729-73c936786914'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:10.151638 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faf0063a11384b9d93d66ad6d378dd2c', 'request_id': 'req-5f0964cc-c8c1-4c45-9729-73c936786914'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:10.157653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faf0063a11384b9d93d66ad6d378dd2c', 'request_id': 'req-5f0964cc-c8c1-4c45-9729-73c936786914'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:10.169518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] The token cache is being invalidate because project faf0063a11384b9d93d66ad6d378dd2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:10.169807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:10.170421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-5f0964cc-c8c1-4c45-9729-73c936786914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:10.171884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1249/2536] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:10 2022] DELETE /identity/v3/projects/faf0063a11384b9d93d66ad6d378dd2c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:10.176827 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e884d-b57e-4ee6-96fb-537db179afca None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.177277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e884d-b57e-4ee6-96fb-537db179afca None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.177373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e884d-b57e-4ee6-96fb-537db179afca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.197496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-731e884d-b57e-4ee6-96fb-537db179afca None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:10.198690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-731e884d-b57e-4ee6-96fb-537db179afca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:10.260970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1288/2537] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:10.269494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e67ee-1494-4ff7-bf3b-f242145a2163 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.269848 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e67ee-1494-4ff7-bf3b-f242145a2163 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.270184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e67ee-1494-4ff7-bf3b-f242145a2163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.291069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-075e67ee-1494-4ff7-bf3b-f242145a2163 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:10.292269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-075e67ee-1494-4ff7-bf3b-f242145a2163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:10.304953 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1250/2538] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:10.309358 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e884d-b57e-4ee6-96fb-537db179afca None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:10.321665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e884d-b57e-4ee6-96fb-537db179afca None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:10.324478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be576b1-ebe5-4b16-b36f-435d7c77ca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:10.325069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be576b1-ebe5-4b16-b36f-435d7c77ca98 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.325345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be576b1-ebe5-4b16-b36f-435d7c77ca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.325716 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be576b1-ebe5-4b16-b36f-435d7c77ca98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.326364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be576b1-ebe5-4b16-b36f-435d7c77ca98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:10.327113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be576b1-ebe5-4b16-b36f-435d7c77ca98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:10.328456 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be576b1-ebe5-4b16-b36f-435d7c77ca98 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:10.342735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1289/2539] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:40:10.344945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e67ee-1494-4ff7-bf3b-f242145a2163 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:10.354388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e67ee-1494-4ff7-bf3b-f242145a2163 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:10.359989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46eaa6-5ef7-4057-9513-9e233dfac742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:10.360601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46eaa6-5ef7-4057-9513-9e233dfac742 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.360857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46eaa6-5ef7-4057-9513-9e233dfac742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.361118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46eaa6-5ef7-4057-9513-9e233dfac742 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.361750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46eaa6-5ef7-4057-9513-9e233dfac742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:10.363602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46eaa6-5ef7-4057-9513-9e233dfac742 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:10.400760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1251/2540] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:10 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:10.402132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be576b1-ebe5-4b16-b36f-435d7c77ca98 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:10.413105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be576b1-ebe5-4b16-b36f-435d7c77ca98 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:10.469667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b6745-ab55-421a-bcb8-db522db8d44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:10.470265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b6745-ab55-421a-bcb8-db522db8d44d admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.470543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b6745-ab55-421a-bcb8-db522db8d44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.470817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b6745-ab55-421a-bcb8-db522db8d44d admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.471442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b6745-ab55-421a-bcb8-db522db8d44d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:10.473393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b6745-ab55-421a-bcb8-db522db8d44d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:10.494345 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1290/2541] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:10 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:10.498536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46eaa6-5ef7-4057-9513-9e233dfac742 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:10.509440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46eaa6-5ef7-4057-9513-9e233dfac742 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:10.513881 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3501efb-c94b-47dd-8fc2-c09ce4957213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:10.514463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3501efb-c94b-47dd-8fc2-c09ce4957213 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.514738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3501efb-c94b-47dd-8fc2-c09ce4957213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.515005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3501efb-c94b-47dd-8fc2-c09ce4957213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.515664 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3501efb-c94b-47dd-8fc2-c09ce4957213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:10.517275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3501efb-c94b-47dd-8fc2-c09ce4957213 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:10.527038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1252/2542] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:10.530954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b6745-ab55-421a-bcb8-db522db8d44d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:10.540995 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b6745-ab55-421a-bcb8-db522db8d44d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:10.545579 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7a696-956e-4c19-a36c-6df4d6d8d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:10.546161 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7a696-956e-4c19-a36c-6df4d6d8d4ee admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.546438 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7a696-956e-4c19-a36c-6df4d6d8d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.547212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7a696-956e-4c19-a36c-6df4d6d8d4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.547843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7a696-956e-4c19-a36c-6df4d6d8d4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:10.549447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7a696-956e-4c19-a36c-6df4d6d8d4ee admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:10.560722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1291/2543] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:10.565266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3501efb-c94b-47dd-8fc2-c09ce4957213 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:10.574972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3501efb-c94b-47dd-8fc2-c09ce4957213 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:10.579531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ef4e2-5882-4dc1-a3fb-e808f6f78d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:10.580165 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ef4e2-5882-4dc1-a3fb-e808f6f78d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.580481 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ef4e2-5882-4dc1-a3fb-e808f6f78d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.580761 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ef4e2-5882-4dc1-a3fb-e808f6f78d1c admin admin] PATH_INFO: `/v3/projects/286157643d4c40968386cc5b18bc5c84/users/16d6729e79794e7886d2cb118104df26/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.594692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ef4e2-5882-4dc1-a3fb-e808f6f78d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=286157643d4c40968386cc5b18bc5c84, user_id=16d6729e79794e7886d2cb118104df26, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:10.597356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ef4e2-5882-4dc1-a3fb-e808f6f78d1c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:10.610175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1253/2544] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:10 2022] PUT /identity/v3/projects/286157643d4c40968386cc5b18bc5c84/users/16d6729e79794e7886d2cb118104df26/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:10.620507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a373975-b03f-44ce-b67c-b5a4b654ae30 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.620796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a373975-b03f-44ce-b67c-b5a4b654ae30 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.621068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a373975-b03f-44ce-b67c-b5a4b654ae30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.648551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0a373975-b03f-44ce-b67c-b5a4b654ae30 None None] MFA Rules not processed for user `16d6729e79794e7886d2cb118104df26`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:10.649752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0a373975-b03f-44ce-b67c-b5a4b654ae30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:10.691910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:10.715233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1292/2545] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:10.720477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ef4e2-5882-4dc1-a3fb-e808f6f78d1c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:10.765135 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:10.784085 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ef4e2-5882-4dc1-a3fb-e808f6f78d1c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:10.789005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb2fc2-bd59-4f1c-8938-86024f8de6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:10.789595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb2fc2-bd59-4f1c-8938-86024f8de6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.789873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb2fc2-bd59-4f1c-8938-86024f8de6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.790145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb2fc2-bd59-4f1c-8938-86024f8de6c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.790771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb2fc2-bd59-4f1c-8938-86024f8de6c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:10.792690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb2fc2-bd59-4f1c-8938-86024f8de6c2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:10.819068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1254/2546] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:10 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:10.823027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a373975-b03f-44ce-b67c-b5a4b654ae30 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:10.833354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a373975-b03f-44ce-b67c-b5a4b654ae30 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:10.886916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b5e8b-facd-4a92-9327-19feeb18bbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:10.887512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b5e8b-facd-4a92-9327-19feeb18bbd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.887786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b5e8b-facd-4a92-9327-19feeb18bbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.888053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b5e8b-facd-4a92-9327-19feeb18bbd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.888706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b5e8b-facd-4a92-9327-19feeb18bbd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:10.890587 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b5e8b-facd-4a92-9327-19feeb18bbd1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:10.911369 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1293/2547] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:10 2022] POST /identity/v3/users => generated 297 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:10.915257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb2fc2-bd59-4f1c-8938-86024f8de6c2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:10.926004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb2fc2-bd59-4f1c-8938-86024f8de6c2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:10.930632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992fd61-7cea-4508-9aec-0006faefa56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:10.931219 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992fd61-7cea-4508-9aec-0006faefa56c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.931492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992fd61-7cea-4508-9aec-0006faefa56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.931759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992fd61-7cea-4508-9aec-0006faefa56c admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.932411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992fd61-7cea-4508-9aec-0006faefa56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:10.933984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992fd61-7cea-4508-9aec-0006faefa56c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:10.944499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1255/2548] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:10.948537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b5e8b-facd-4a92-9327-19feeb18bbd1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:10.958264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b5e8b-facd-4a92-9327-19feeb18bbd1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:10.962732 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f5cc9-e1c9-4f9f-9ea6-f856c3ec6b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:10.963361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f5cc9-e1c9-4f9f-9ea6-f856c3ec6b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:10.963635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f5cc9-e1c9-4f9f-9ea6-f856c3ec6b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:10.963907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f5cc9-e1c9-4f9f-9ea6-f856c3ec6b52 admin admin] PATH_INFO: `/v3/projects/f8dd8527e2fb487cadd14b1935e653d9/users/cf1cc2a4e8964f8a9d1119a4db7c1325/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:10.978505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f5cc9-e1c9-4f9f-9ea6-f856c3ec6b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8dd8527e2fb487cadd14b1935e653d9, user_id=cf1cc2a4e8964f8a9d1119a4db7c1325, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:10.981150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f5cc9-e1c9-4f9f-9ea6-f856c3ec6b52 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:10.994001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1294/2549] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:10 2022] PUT /identity/v3/projects/f8dd8527e2fb487cadd14b1935e653d9/users/cf1cc2a4e8964f8a9d1119a4db7c1325/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:10.997896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992fd61-7cea-4508-9aec-0006faefa56c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:11.007812 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992fd61-7cea-4508-9aec-0006faefa56c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:11.066980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7853a8-b3a0-404f-b128-b00e67762911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:11.067504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7853a8-b3a0-404f-b128-b00e67762911 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:11.067796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7853a8-b3a0-404f-b128-b00e67762911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:11.068051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7853a8-b3a0-404f-b128-b00e67762911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:11.068721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7853a8-b3a0-404f-b128-b00e67762911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:11.070323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7853a8-b3a0-404f-b128-b00e67762911 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:11.080350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1256/2550] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:11.084545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f5cc9-e1c9-4f9f-9ea6-f856c3ec6b52 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:11.095112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f5cc9-e1c9-4f9f-9ea6-f856c3ec6b52 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:11.099652 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49a43f-7498-4bcd-8f6c-da5475571c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:11.100290 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49a43f-7498-4bcd-8f6c-da5475571c44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:11.100605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49a43f-7498-4bcd-8f6c-da5475571c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:11.100873 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49a43f-7498-4bcd-8f6c-da5475571c44 admin admin] PATH_INFO: `/v3/projects/f8dd8527e2fb487cadd14b1935e653d9/users/cf1cc2a4e8964f8a9d1119a4db7c1325/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:11.102760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49a43f-7498-4bcd-8f6c-da5475571c44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8dd8527e2fb487cadd14b1935e653d9, user_id=cf1cc2a4e8964f8a9d1119a4db7c1325, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:11.105446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49a43f-7498-4bcd-8f6c-da5475571c44 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:11.115203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1295/2551] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:11 2022] PUT /identity/v3/projects/f8dd8527e2fb487cadd14b1935e653d9/users/cf1cc2a4e8964f8a9d1119a4db7c1325/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:11.125922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e54d5-3718-4b45-8ea7-b6d8f2332b8d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:11.126410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e54d5-3718-4b45-8ea7-b6d8f2332b8d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:11.126531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e54d5-3718-4b45-8ea7-b6d8f2332b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:11.153860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-081e54d5-3718-4b45-8ea7-b6d8f2332b8d None None] MFA Rules not processed for user `cf1cc2a4e8964f8a9d1119a4db7c1325`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:11.155040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-081e54d5-3718-4b45-8ea7-b6d8f2332b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:11.221327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1257/2552] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:11.232036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49a43f-7498-4bcd-8f6c-da5475571c44 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:11.233775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0a49a43f-7498-4bcd-8f6c-da5475571c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:11.289331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49a43f-7498-4bcd-8f6c-da5475571c44 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:11.292904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadd735-a343-4cf3-8b43-f368460c4cd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:11.293490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadd735-a343-4cf3-8b43-f368460c4cd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:11.293760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadd735-a343-4cf3-8b43-f368460c4cd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:11.294023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadd735-a343-4cf3-8b43-f368460c4cd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:11.302603 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baadd735-a343-4cf3-8b43-f368460c4cd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:11.304000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baadd735-a343-4cf3-8b43-f368460c4cd1 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:11.312962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1296/2553] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:11.708319 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e54d5-3718-4b45-8ea7-b6d8f2332b8d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:11.754814 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:11.777043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e54d5-3718-4b45-8ea7-b6d8f2332b8d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:11.781524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:11.782143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:11.782419 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:11.782688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] PATH_INFO: `/v3/users/16d6729e79794e7886d2cb118104df26` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:11.783791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d6729e79794e7886d2cb118104df26)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:11.785986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:11.830579 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d6729e79794e7886d2cb118104df26'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:11.837303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d6729e79794e7886d2cb118104df26'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:11.837950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d6729e79794e7886d2cb118104df26'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:11.844912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d6729e79794e7886d2cb118104df26'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:11.856855 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d6729e79794e7886d2cb118104df26'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:11.863249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1258/2554] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:11 2022] DELETE /identity/v3/users/16d6729e79794e7886d2cb118104df26 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:11.867606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1696e9-ba47-4120-9f9a-917d87217824 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:11.867935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1696e9-ba47-4120-9f9a-917d87217824 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:11.868226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1696e9-ba47-4120-9f9a-917d87217824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:11.886977 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-3e1696e9-ba47-4120-9f9a-917d87217824 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:11.888147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3e1696e9-ba47-4120-9f9a-917d87217824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:11.922526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:11.954987 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1297/2555] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:11.961902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:11.963294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:12.021108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033149e-d21b-4b7b-9909-bebd9b8139fb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:12.024769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09838ff8-1856-4ca7-9924-7ce88d1d340c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:12.025363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838ff8-1856-4ca7-9924-7ce88d1d340c service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:12.025637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838ff8-1856-4ca7-9924-7ce88d1d340c service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:12.025903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09838ff8-1856-4ca7-9924-7ce88d1d340c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:12.034974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09838ff8-1856-4ca7-9924-7ce88d1d340c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:12.036806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09838ff8-1856-4ca7-9924-7ce88d1d340c service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:12.046279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1259/2556] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:12.308809 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1696e9-ba47-4120-9f9a-917d87217824 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:12.310350 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3e1696e9-ba47-4120-9f9a-917d87217824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:12.325259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1696e9-ba47-4120-9f9a-917d87217824 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:12.330837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:12.331292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:12.331608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:12.331831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] PATH_INFO: `/v3/projects/286157643d4c40968386cc5b18bc5c84` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:12.332974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=286157643d4c40968386cc5b18bc5c84)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:12.334856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:12.380762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '286157643d4c40968386cc5b18bc5c84', 'request_id': 'req-fd1a694a-8abb-4931-bea0-99c18b95ada1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.386276 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '286157643d4c40968386cc5b18bc5c84', 'request_id': 'req-fd1a694a-8abb-4931-bea0-99c18b95ada1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.393301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '286157643d4c40968386cc5b18bc5c84', 'request_id': 'req-fd1a694a-8abb-4931-bea0-99c18b95ada1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.406626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] The token cache is being invalidate because project 286157643d4c40968386cc5b18bc5c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:12.406914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.407534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.409024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1298/2557] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:12 2022] DELETE /identity/v3/projects/286157643d4c40968386cc5b18bc5c84 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:12.413280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09838ff8-1856-4ca7-9924-7ce88d1d340c service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:12.423561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09838ff8-1856-4ca7-9924-7ce88d1d340c service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:12.478203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:12.478820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:12.479094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:12.479356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] PATH_INFO: `/v3/users/cf1cc2a4e8964f8a9d1119a4db7c1325` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:12.480482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1cc2a4e8964f8a9d1119a4db7c1325)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:12.482365 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:12.483762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:12.536156 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1cc2a4e8964f8a9d1119a4db7c1325'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.543014 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1cc2a4e8964f8a9d1119a4db7c1325'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.543663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1cc2a4e8964f8a9d1119a4db7c1325'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.550146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1cc2a4e8964f8a9d1119a4db7c1325'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.563144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1cc2a4e8964f8a9d1119a4db7c1325'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.568909 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1260/2558] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:12 2022] DELETE /identity/v3/users/cf1cc2a4e8964f8a9d1119a4db7c1325 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:12.831451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:12.833144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:12.840461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:12.842499 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:12.903158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a694a-8abb-4931-bea0-99c18b95ada1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:12.910569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e39b5-74ad-4126-ae83-fac9a0683cc6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:12.915154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:12.915709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:12.915981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:12.916250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] PATH_INFO: `/v3/projects/f8dd8527e2fb487cadd14b1935e653d9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:12.917368 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8dd8527e2fb487cadd14b1935e653d9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:12.919249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:12.953755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39489a1d-60e9-471a-9c28-b74ca01c76a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:12.954366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39489a1d-60e9-471a-9c28-b74ca01c76a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:12.954658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39489a1d-60e9-471a-9c28-b74ca01c76a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:12.954908 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39489a1d-60e9-471a-9c28-b74ca01c76a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:12.956677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-39489a1d-60e9-471a-9c28-b74ca01c76a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:12.967145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8dd8527e2fb487cadd14b1935e653d9', 'request_id': 'req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.968645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39489a1d-60e9-471a-9c28-b74ca01c76a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:12.970062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39489a1d-60e9-471a-9c28-b74ca01c76a5 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:12.972568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8dd8527e2fb487cadd14b1935e653d9', 'request_id': 'req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.978659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8dd8527e2fb487cadd14b1935e653d9', 'request_id': 'req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.979749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1299/2559] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:12.991334 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] The token cache is being invalidate because project f8dd8527e2fb487cadd14b1935e653d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:12.991759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.992406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-87dccf5e-676c-4ad4-8cb9-e07a5c74bce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:12.993955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1261/2560] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:12 2022] DELETE /identity/v3/projects/f8dd8527e2fb487cadd14b1935e653d9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:12.998933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f1c7-7c09-4886-9b9e-12e5550a8c55 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:12.999218 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f1c7-7c09-4886-9b9e-12e5550a8c55 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:12.999489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f1c7-7c09-4886-9b9e-12e5550a8c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:13.020503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-50c1f1c7-7c09-4886-9b9e-12e5550a8c55 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:13.021699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-50c1f1c7-7c09-4886-9b9e-12e5550a8c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:13.034466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:13.082206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1300/2561] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:13.090544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd730-34f1-4a8c-9616-fd2c71eed86d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:13.090853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd730-34f1-4a8c-9616-fd2c71eed86d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:13.091096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd730-34f1-4a8c-9616-fd2c71eed86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:13.110674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-2bfbd730-34f1-4a8c-9616-fd2c71eed86d None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:13.111986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2bfbd730-34f1-4a8c-9616-fd2c71eed86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:13.119805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1262/2562] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:13.124044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1f1c7-7c09-4886-9b9e-12e5550a8c55 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:13.134311 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1f1c7-7c09-4886-9b9e-12e5550a8c55 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:13.139159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6233c86-bfe1-47ad-a224-ef28d3344256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:13.139752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6233c86-bfe1-47ad-a224-ef28d3344256 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:13.140026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6233c86-bfe1-47ad-a224-ef28d3344256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:13.140290 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6233c86-bfe1-47ad-a224-ef28d3344256 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:13.141528 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6233c86-bfe1-47ad-a224-ef28d3344256 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:13.141826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6233c86-bfe1-47ad-a224-ef28d3344256 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:13.143394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6233c86-bfe1-47ad-a224-ef28d3344256 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:13.154765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1301/2563] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:40:13.159242 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbd730-34f1-4a8c-9616-fd2c71eed86d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:13.163851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:13.169863 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbd730-34f1-4a8c-9616-fd2c71eed86d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:13.175948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878262c-408c-405b-b088-ccbd2e32b501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:13.176586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878262c-408c-405b-b088-ccbd2e32b501 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:13.176864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878262c-408c-405b-b088-ccbd2e32b501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:13.177129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878262c-408c-405b-b088-ccbd2e32b501 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:13.177751 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878262c-408c-405b-b088-ccbd2e32b501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:13.179648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878262c-408c-405b-b088-ccbd2e32b501 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:13.207833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1263/2564] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:13 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:40:13.212098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6233c86-bfe1-47ad-a224-ef28d3344256 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:13.221861 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6233c86-bfe1-47ad-a224-ef28d3344256 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:13.277926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba597d-65de-4a7d-88d1-40d2b326496e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:13.278530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba597d-65de-4a7d-88d1-40d2b326496e admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:13.278801 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba597d-65de-4a7d-88d1-40d2b326496e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:13.279068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba597d-65de-4a7d-88d1-40d2b326496e admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:13.279679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba597d-65de-4a7d-88d1-40d2b326496e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:13.281631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba597d-65de-4a7d-88d1-40d2b326496e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:13.303850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1302/2565] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:13 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:13.307871 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878262c-408c-405b-b088-ccbd2e32b501 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:13.318483 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878262c-408c-405b-b088-ccbd2e32b501 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:13.323032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e76985-fec7-41cf-bd24-0081d7ee3344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:13.323618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76985-fec7-41cf-bd24-0081d7ee3344 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:13.323890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76985-fec7-41cf-bd24-0081d7ee3344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:13.324157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76985-fec7-41cf-bd24-0081d7ee3344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:13.324831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e76985-fec7-41cf-bd24-0081d7ee3344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:13.326452 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e76985-fec7-41cf-bd24-0081d7ee3344 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:13.336139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1264/2566] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:40:13.340209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba597d-65de-4a7d-88d1-40d2b326496e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:13.350272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba597d-65de-4a7d-88d1-40d2b326496e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:13.355719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2856d5-9419-42d5-8393-8bd51e3f8571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:13.356315 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2856d5-9419-42d5-8393-8bd51e3f8571 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:13.356629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2856d5-9419-42d5-8393-8bd51e3f8571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:13.356900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2856d5-9419-42d5-8393-8bd51e3f8571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:13.357538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2856d5-9419-42d5-8393-8bd51e3f8571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:13.359679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2856d5-9419-42d5-8393-8bd51e3f8571 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:13.370479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1303/2567] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:13.374464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e76985-fec7-41cf-bd24-0081d7ee3344 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:13.385684 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e76985-fec7-41cf-bd24-0081d7ee3344 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:13.391857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ade0cd-ea60-4eae-a76a-9c4386693b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:13.392540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ade0cd-ea60-4eae-a76a-9c4386693b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:13.392820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ade0cd-ea60-4eae-a76a-9c4386693b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:13.393092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ade0cd-ea60-4eae-a76a-9c4386693b50 admin admin] PATH_INFO: `/v3/projects/e503204069374989bd14c21845435f88/users/a6ef75911b594356a5c5685e7b7a2792/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:13.408594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ade0cd-ea60-4eae-a76a-9c4386693b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e503204069374989bd14c21845435f88, user_id=a6ef75911b594356a5c5685e7b7a2792, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:13.411622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ade0cd-ea60-4eae-a76a-9c4386693b50 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:13.426071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1265/2568] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:13 2022] PUT /identity/v3/projects/e503204069374989bd14c21845435f88/users/a6ef75911b594356a5c5685e7b7a2792/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:13.438127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0c86b-d51a-4ace-bbbd-431a5c381f3f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:13.438413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0c86b-d51a-4ace-bbbd-431a5c381f3f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:13.438680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0c86b-d51a-4ace-bbbd-431a5c381f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:13.465945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ffa0c86b-d51a-4ace-bbbd-431a5c381f3f None None] MFA Rules not processed for user `a6ef75911b594356a5c5685e7b7a2792`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:13.467148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ffa0c86b-d51a-4ace-bbbd-431a5c381f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:13.531056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1304/2569] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:13.542264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ade0cd-ea60-4eae-a76a-9c4386693b50 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:13.543925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-49ade0cd-ea60-4eae-a76a-9c4386693b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:13.601493 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ade0cd-ea60-4eae-a76a-9c4386693b50 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:13.604954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f5ffc-e124-4e32-aa3f-da5e6aa3bdee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:13.605550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f5ffc-e124-4e32-aa3f-da5e6aa3bdee service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:13.605826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f5ffc-e124-4e32-aa3f-da5e6aa3bdee service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:13.606094 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f5ffc-e124-4e32-aa3f-da5e6aa3bdee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:13.615039 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f5ffc-e124-4e32-aa3f-da5e6aa3bdee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:13.616548 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f5ffc-e124-4e32-aa3f-da5e6aa3bdee service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:13.625903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1266/2570] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:16.105961 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0c86b-d51a-4ace-bbbd-431a5c381f3f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:16.107437 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ffa0c86b-d51a-4ace-bbbd-431a5c381f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:16.127158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:16.177657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0c86b-d51a-4ace-bbbd-431a5c381f3f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:16.182121 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37fd8c-f410-401d-a75f-52682b586c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:16.182745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37fd8c-f410-401d-a75f-52682b586c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:16.183021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37fd8c-f410-401d-a75f-52682b586c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:16.183291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37fd8c-f410-401d-a75f-52682b586c22 admin admin] PATH_INFO: `/v3/users/ccbdadc310fc4bf8a6679bcb39b34ff1` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:16.197291 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37fd8c-f410-401d-a75f-52682b586c22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccbdadc310fc4bf8a6679bcb39b34ff1)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:16.199201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37fd8c-f410-401d-a75f-52682b586c22 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:16.246501 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7d37fd8c-f410-401d-a75f-52682b586c22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccbdadc310fc4bf8a6679bcb39b34ff1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:16.254017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7d37fd8c-f410-401d-a75f-52682b586c22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccbdadc310fc4bf8a6679bcb39b34ff1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:16.254676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7d37fd8c-f410-401d-a75f-52682b586c22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccbdadc310fc4bf8a6679bcb39b34ff1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:16.261932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7d37fd8c-f410-401d-a75f-52682b586c22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccbdadc310fc4bf8a6679bcb39b34ff1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:16.274144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7d37fd8c-f410-401d-a75f-52682b586c22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccbdadc310fc4bf8a6679bcb39b34ff1'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:16.279838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1305/2571] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:16 2022] DELETE /identity/v3/users/ccbdadc310fc4bf8a6679bcb39b34ff1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:16.452241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f5ffc-e124-4e32-aa3f-da5e6aa3bdee service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:16.463605 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f5ffc-e124-4e32-aa3f-da5e6aa3bdee service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:16.520433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:16.521057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:16.521344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:16.521613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] PATH_INFO: `/v3/projects/612fb8a339e4478ba78bc5ef3b343b82` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:16.531770 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=612fb8a339e4478ba78bc5ef3b343b82)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:16.533689 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:16.579850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:16.589429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '612fb8a339e4478ba78bc5ef3b343b82', 'request_id': 'req-b0d4e347-6744-4445-833d-6925a56e6484'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:16.595506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '612fb8a339e4478ba78bc5ef3b343b82', 'request_id': 'req-b0d4e347-6744-4445-833d-6925a56e6484'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:16.602154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '612fb8a339e4478ba78bc5ef3b343b82', 'request_id': 'req-b0d4e347-6744-4445-833d-6925a56e6484'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:16.615175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] The token cache is being invalidate because project 612fb8a339e4478ba78bc5ef3b343b82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:16.615495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:16.616131 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b0d4e347-6744-4445-833d-6925a56e6484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:16.617824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1267/2572] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:16 2022] DELETE /identity/v3/projects/612fb8a339e4478ba78bc5ef3b343b82 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:16.624030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797059b8-6978-4c39-a203-5c34d03bfc61 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:16.624292 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797059b8-6978-4c39-a203-5c34d03bfc61 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:16.624507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797059b8-6978-4c39-a203-5c34d03bfc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:16.652062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-797059b8-6978-4c39-a203-5c34d03bfc61 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:16.654184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-797059b8-6978-4c39-a203-5c34d03bfc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:16.743733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1306/2573] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:16.751642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca9e8-088c-46ba-ab36-814fa80dba5f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:16.751968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca9e8-088c-46ba-ab36-814fa80dba5f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:16.752241 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca9e8-088c-46ba-ab36-814fa80dba5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:16.772218 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-baeca9e8-088c-46ba-ab36-814fa80dba5f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:16.773446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-baeca9e8-088c-46ba-ab36-814fa80dba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:16.781566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1268/2574] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:16.786132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797059b8-6978-4c39-a203-5c34d03bfc61 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:16.795737 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797059b8-6978-4c39-a203-5c34d03bfc61 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:16.800886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8335d-8406-4442-b925-9274158a10d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:16.801476 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8335d-8406-4442-b925-9274158a10d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:16.801751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8335d-8406-4442-b925-9274158a10d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:16.802018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8335d-8406-4442-b925-9274158a10d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:16.802692 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8335d-8406-4442-b925-9274158a10d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:16.802990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8335d-8406-4442-b925-9274158a10d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:16.804674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8335d-8406-4442-b925-9274158a10d5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:16.815930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1307/2575] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:40:16.822228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca9e8-088c-46ba-ab36-814fa80dba5f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:16.832003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca9e8-088c-46ba-ab36-814fa80dba5f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:16.836819 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4641a2-7ae0-4a47-abd2-9ab6e7c1ac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:16.837465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4641a2-7ae0-4a47-abd2-9ab6e7c1ac02 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:16.837740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4641a2-7ae0-4a47-abd2-9ab6e7c1ac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:16.838007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4641a2-7ae0-4a47-abd2-9ab6e7c1ac02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:16.838626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4641a2-7ae0-4a47-abd2-9ab6e7c1ac02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:16.840578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4641a2-7ae0-4a47-abd2-9ab6e7c1ac02 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:16.868244 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1269/2576] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:16 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:16.872518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8335d-8406-4442-b925-9274158a10d5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:16.882306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8335d-8406-4442-b925-9274158a10d5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:16.890028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:16.939527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf9ae1-2e61-440b-82fc-dada0e6c0880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:16.940112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf9ae1-2e61-440b-82fc-dada0e6c0880 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:16.940409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf9ae1-2e61-440b-82fc-dada0e6c0880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:16.940680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf9ae1-2e61-440b-82fc-dada0e6c0880 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:16.941288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf9ae1-2e61-440b-82fc-dada0e6c0880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:16.943168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf9ae1-2e61-440b-82fc-dada0e6c0880 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:16.962594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1308/2577] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:16 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:16.966788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4641a2-7ae0-4a47-abd2-9ab6e7c1ac02 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:16.977374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4641a2-7ae0-4a47-abd2-9ab6e7c1ac02 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:16.981837 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4421c6-ab21-4f6e-a8bd-3f4d89545f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:16.982423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4421c6-ab21-4f6e-a8bd-3f4d89545f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:16.982697 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4421c6-ab21-4f6e-a8bd-3f4d89545f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:16.982962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4421c6-ab21-4f6e-a8bd-3f4d89545f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:16.983588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4421c6-ab21-4f6e-a8bd-3f4d89545f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:16.985753 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4421c6-ab21-4f6e-a8bd-3f4d89545f48 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:16.995756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1270/2578] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:16.999558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf9ae1-2e61-440b-82fc-dada0e6c0880 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:17.009206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf9ae1-2e61-440b-82fc-dada0e6c0880 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:17.013748 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68cd582-ce1b-4b7e-b6f9-3247d2077256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:17.014330 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cd582-ce1b-4b7e-b6f9-3247d2077256 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:17.014604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cd582-ce1b-4b7e-b6f9-3247d2077256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:17.014870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cd582-ce1b-4b7e-b6f9-3247d2077256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:17.016024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68cd582-ce1b-4b7e-b6f9-3247d2077256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:17.017671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68cd582-ce1b-4b7e-b6f9-3247d2077256 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:17.027878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1309/2579] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:17.032549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4421c6-ab21-4f6e-a8bd-3f4d89545f48 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:17.042843 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4421c6-ab21-4f6e-a8bd-3f4d89545f48 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:17.047285 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169a9dd-4da2-4d66-811e-5f3585b25a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:17.047913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a9dd-4da2-4d66-811e-5f3585b25a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:17.048186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a9dd-4da2-4d66-811e-5f3585b25a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:17.048495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a9dd-4da2-4d66-811e-5f3585b25a29 admin admin] PATH_INFO: `/v3/projects/0167964b2baf4d50962d5cfcf4f36573/users/b5f2f1da75cd4111ad0bef7c342167aa/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:17.062246 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169a9dd-4da2-4d66-811e-5f3585b25a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0167964b2baf4d50962d5cfcf4f36573, user_id=b5f2f1da75cd4111ad0bef7c342167aa, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:17.064897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169a9dd-4da2-4d66-811e-5f3585b25a29 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:17.077121 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1271/2580] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:17 2022] PUT /identity/v3/projects/0167964b2baf4d50962d5cfcf4f36573/users/b5f2f1da75cd4111ad0bef7c342167aa/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:17.087507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3fef6-8ccb-45c8-8364-3979078c05ea None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:17.087789 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3fef6-8ccb-45c8-8364-3979078c05ea None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:17.088066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3fef6-8ccb-45c8-8364-3979078c05ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:17.116591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-fbf3fef6-8ccb-45c8-8364-3979078c05ea None None] MFA Rules not processed for user `b5f2f1da75cd4111ad0bef7c342167aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:17.117817 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-fbf3fef6-8ccb-45c8-8364-3979078c05ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:17.179717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1310/2581] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:17.184040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169a9dd-4da2-4d66-811e-5f3585b25a29 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:17.246282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169a9dd-4da2-4d66-811e-5f3585b25a29 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:17.250981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d956b-41ec-4a70-8d66-0dffd1f08804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:17.251580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d956b-41ec-4a70-8d66-0dffd1f08804 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:17.251845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d956b-41ec-4a70-8d66-0dffd1f08804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:17.252115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d956b-41ec-4a70-8d66-0dffd1f08804 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:17.252782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d956b-41ec-4a70-8d66-0dffd1f08804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:17.254657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d956b-41ec-4a70-8d66-0dffd1f08804 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:17.287993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1272/2582] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:17 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:17.291959 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3fef6-8ccb-45c8-8364-3979078c05ea None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:17.303924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3fef6-8ccb-45c8-8364-3979078c05ea None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:17.361281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff17da8-93fd-44f5-9f25-daa6fdfb1132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:17.361895 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff17da8-93fd-44f5-9f25-daa6fdfb1132 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:17.362173 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff17da8-93fd-44f5-9f25-daa6fdfb1132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:17.362442 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff17da8-93fd-44f5-9f25-daa6fdfb1132 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:17.363062 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff17da8-93fd-44f5-9f25-daa6fdfb1132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:17.364996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff17da8-93fd-44f5-9f25-daa6fdfb1132 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:17.385946 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1311/2583] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:17 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:17.389891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d956b-41ec-4a70-8d66-0dffd1f08804 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:17.400657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d956b-41ec-4a70-8d66-0dffd1f08804 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:17.405199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795166fe-934e-4bea-be4b-f22bfec4fa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:17.405784 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795166fe-934e-4bea-be4b-f22bfec4fa44 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:17.406124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795166fe-934e-4bea-be4b-f22bfec4fa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:17.406393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795166fe-934e-4bea-be4b-f22bfec4fa44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:17.407023 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795166fe-934e-4bea-be4b-f22bfec4fa44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:17.408659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795166fe-934e-4bea-be4b-f22bfec4fa44 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:17.419481 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1273/2584] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:17.423544 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff17da8-93fd-44f5-9f25-daa6fdfb1132 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:17.433605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff17da8-93fd-44f5-9f25-daa6fdfb1132 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:17.438642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea782ab-0092-42a4-906f-7ebdd9728953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:17.439286 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea782ab-0092-42a4-906f-7ebdd9728953 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:17.439562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea782ab-0092-42a4-906f-7ebdd9728953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:17.439830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea782ab-0092-42a4-906f-7ebdd9728953 admin admin] PATH_INFO: `/v3/projects/10b16fc4b46943c297cda34073249520/users/ee3d2f037bd04f02901af78562b0ea12/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:17.442805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:17.455807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea782ab-0092-42a4-906f-7ebdd9728953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b16fc4b46943c297cda34073249520, user_id=ee3d2f037bd04f02901af78562b0ea12, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:17.458460 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea782ab-0092-42a4-906f-7ebdd9728953 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:17.471140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1312/2585] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:17 2022] PUT /identity/v3/projects/10b16fc4b46943c297cda34073249520/users/ee3d2f037bd04f02901af78562b0ea12/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:17.475158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795166fe-934e-4bea-be4b-f22bfec4fa44 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:17.486993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795166fe-934e-4bea-be4b-f22bfec4fa44 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:17.523227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:17.543477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9979765-eb8d-41cf-b8d4-306d9c5cb380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:17.544074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9979765-eb8d-41cf-b8d4-306d9c5cb380 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:17.544353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9979765-eb8d-41cf-b8d4-306d9c5cb380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:17.544660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9979765-eb8d-41cf-b8d4-306d9c5cb380 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:17.545294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9979765-eb8d-41cf-b8d4-306d9c5cb380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:17.547408 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9979765-eb8d-41cf-b8d4-306d9c5cb380 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:17.556834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1274/2586] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:17.560951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea782ab-0092-42a4-906f-7ebdd9728953 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:17.571036 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea782ab-0092-42a4-906f-7ebdd9728953 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:17.575568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6725a4f-18a5-4c91-b8d2-a2bf777c8d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:17.576202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6725a4f-18a5-4c91-b8d2-a2bf777c8d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:17.576510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6725a4f-18a5-4c91-b8d2-a2bf777c8d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:17.576781 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6725a4f-18a5-4c91-b8d2-a2bf777c8d8a admin admin] PATH_INFO: `/v3/projects/10b16fc4b46943c297cda34073249520/users/ee3d2f037bd04f02901af78562b0ea12/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:17.578683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6725a4f-18a5-4c91-b8d2-a2bf777c8d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b16fc4b46943c297cda34073249520, user_id=ee3d2f037bd04f02901af78562b0ea12, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:17.581300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6725a4f-18a5-4c91-b8d2-a2bf777c8d8a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:17.591098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1313/2587] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:17 2022] PUT /identity/v3/projects/10b16fc4b46943c297cda34073249520/users/ee3d2f037bd04f02901af78562b0ea12/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:17.602677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb37cc-49b1-4d8d-8174-ec49914778e9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:17.602984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb37cc-49b1-4d8d-8174-ec49914778e9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:17.603239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb37cc-49b1-4d8d-8174-ec49914778e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:17.631089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dfcb37cc-49b1-4d8d-8174-ec49914778e9 None None] MFA Rules not processed for user `ee3d2f037bd04f02901af78562b0ea12`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:17.632281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dfcb37cc-49b1-4d8d-8174-ec49914778e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:17.699181 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1275/2588] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:17.711311 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6725a4f-18a5-4c91-b8d2-a2bf777c8d8a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:17.713027 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d6725a4f-18a5-4c91-b8d2-a2bf777c8d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:17.775150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6725a4f-18a5-4c91-b8d2-a2bf777c8d8a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:17.778831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388d4aa-ed34-4b03-9715-fd72a5210177 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:17.779429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d4aa-ed34-4b03-9715-fd72a5210177 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:17.779709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d4aa-ed34-4b03-9715-fd72a5210177 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:17.779978 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d4aa-ed34-4b03-9715-fd72a5210177 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:17.790459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388d4aa-ed34-4b03-9715-fd72a5210177 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:17.791879 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388d4aa-ed34-4b03-9715-fd72a5210177 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:17.801224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1314/2589] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:18.104442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb37cc-49b1-4d8d-8174-ec49914778e9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:18.113339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb37cc-49b1-4d8d-8174-ec49914778e9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:18.117787 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981f888-d92c-496f-933b-5a04add8a0ce tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee3d2f037bd04f02901af78562b0ea12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10b16fc4b46943c297cda34073249520', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:18.118377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981f888-d92c-496f-933b-5a04add8a0ce tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:18.118652 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981f888-d92c-496f-933b-5a04add8a0ce tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:18.118919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981f888-d92c-496f-933b-5a04add8a0ce tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:18.119553 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981f888-d92c-496f-933b-5a04add8a0ce tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:18.119851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981f888-d92c-496f-933b-5a04add8a0ce tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:18.121459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981f888-d92c-496f-933b-5a04add8a0ce tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:18.132968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1276/2590] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:40:18.136999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388d4aa-ed34-4b03-9715-fd72a5210177 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:18.146560 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388d4aa-ed34-4b03-9715-fd72a5210177 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:18.151146 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c39e4-e49f-410b-a00b-5515b1693901 tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee3d2f037bd04f02901af78562b0ea12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10b16fc4b46943c297cda34073249520', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:18.151741 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c39e4-e49f-410b-a00b-5515b1693901 tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:18.152022 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c39e4-e49f-410b-a00b-5515b1693901 tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:18.152296 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c39e4-e49f-410b-a00b-5515b1693901 tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:18.152955 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c39e4-e49f-410b-a00b-5515b1693901 tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:18.154814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c39e4-e49f-410b-a00b-5515b1693901 tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:18.180609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1315/2591] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:18 2022] POST /identity/v3/projects => generated 355 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:18.184529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981f888-d92c-496f-933b-5a04add8a0ce tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:18.195353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981f888-d92c-496f-933b-5a04add8a0ce tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:18.250274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97a673-de8a-4027-b133-300fa3f5247a tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee3d2f037bd04f02901af78562b0ea12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10b16fc4b46943c297cda34073249520', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:18.250863 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97a673-de8a-4027-b133-300fa3f5247a tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:18.251142 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97a673-de8a-4027-b133-300fa3f5247a tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:18.251406 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97a673-de8a-4027-b133-300fa3f5247a tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:18.252038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97a673-de8a-4027-b133-300fa3f5247a tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:18.252333 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97a673-de8a-4027-b133-300fa3f5247a tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:18.253932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97a673-de8a-4027-b133-300fa3f5247a tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:18.265874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1277/2592] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:40:18.304866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c39e4-e49f-410b-a00b-5515b1693901 tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:18.315049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c39e4-e49f-410b-a00b-5515b1693901 tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:18.319425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee3d2f037bd04f02901af78562b0ea12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10b16fc4b46943c297cda34073249520', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:18.320042 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:18.320462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:18.320738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] PATH_INFO: `/v3/projects/8cb99ca15b5f48b096b128d3f5a76bb7` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:18.321857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8cb99ca15b5f48b096b128d3f5a76bb7)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:18.323846 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:18.372568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cb99ca15b5f48b096b128d3f5a76bb7', 'request_id': 'req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:18.378245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cb99ca15b5f48b096b128d3f5a76bb7', 'request_id': 'req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:18.384810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cb99ca15b5f48b096b128d3f5a76bb7', 'request_id': 'req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:18.397094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] The token cache is being invalidate because project 8cb99ca15b5f48b096b128d3f5a76bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:18.397375 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:18.398017 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2b7aaf0-c53c-45c7-893c-e5ed7f2f73de tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:18.399479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1316/2593] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:18 2022] DELETE /identity/v3/projects/8cb99ca15b5f48b096b128d3f5a76bb7 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:18.584593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97a673-de8a-4027-b133-300fa3f5247a tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:18.586350 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5f97a673-de8a-4027-b133-300fa3f5247a tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:18.630508 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:18.650629 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97a673-de8a-4027-b133-300fa3f5247a tempest-VolumeQuotaClassesTest-207980743 tempest-VolumeQuotaClassesTest-207980743-project-admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:18.705317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:18.705936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:18.706209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:18.706476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] PATH_INFO: `/v3/users/b5f2f1da75cd4111ad0bef7c342167aa` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:18.707607 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f2f1da75cd4111ad0bef7c342167aa)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:18.709540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:18.754458 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f2f1da75cd4111ad0bef7c342167aa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:18.761329 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f2f1da75cd4111ad0bef7c342167aa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:18.761972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f2f1da75cd4111ad0bef7c342167aa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:18.769515 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f2f1da75cd4111ad0bef7c342167aa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:18.781042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f2f1da75cd4111ad0bef7c342167aa'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:18.787536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1278/2594] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:18 2022] DELETE /identity/v3/users/b5f2f1da75cd4111ad0bef7c342167aa => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:18.792032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac8221-9080-4b3e-bf4f-96414f7de096 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:18.792332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac8221-9080-4b3e-bf4f-96414f7de096 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:18.792626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac8221-9080-4b3e-bf4f-96414f7de096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:18.810553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-40ac8221-9080-4b3e-bf4f-96414f7de096 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:18.811717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-40ac8221-9080-4b3e-bf4f-96414f7de096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:18.841339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:18.872775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1317/2595] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:18.883008 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:18.884409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:18.941550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea446f41-48e0-401e-9bd3-871e153a9b09 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:18.945164 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db03630-1bc2-474f-8851-c5e995862d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:18.945767 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db03630-1bc2-474f-8851-c5e995862d11 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:18.946046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db03630-1bc2-474f-8851-c5e995862d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:18.946314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db03630-1bc2-474f-8851-c5e995862d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:18.955854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db03630-1bc2-474f-8851-c5e995862d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:18.957322 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db03630-1bc2-474f-8851-c5e995862d11 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:18.966525 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1279/2596] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:19.232086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac8221-9080-4b3e-bf4f-96414f7de096 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:19.233621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-40ac8221-9080-4b3e-bf4f-96414f7de096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:19.246288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac8221-9080-4b3e-bf4f-96414f7de096 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:19.252092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:19.252750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:19.253032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:19.253299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] PATH_INFO: `/v3/projects/0167964b2baf4d50962d5cfcf4f36573` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:19.254419 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0167964b2baf4d50962d5cfcf4f36573)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:19.256502 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:19.303024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0167964b2baf4d50962d5cfcf4f36573', 'request_id': 'req-46fcb93b-dc36-4545-84b0-ae10b231d93e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.309589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0167964b2baf4d50962d5cfcf4f36573', 'request_id': 'req-46fcb93b-dc36-4545-84b0-ae10b231d93e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.316574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0167964b2baf4d50962d5cfcf4f36573', 'request_id': 'req-46fcb93b-dc36-4545-84b0-ae10b231d93e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.330526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] The token cache is being invalidate because project 0167964b2baf4d50962d5cfcf4f36573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:19.331142 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.331889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.333497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1318/2597] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:19 2022] DELETE /identity/v3/projects/0167964b2baf4d50962d5cfcf4f36573 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:19.337486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db03630-1bc2-474f-8851-c5e995862d11 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:19.346885 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db03630-1bc2-474f-8851-c5e995862d11 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:19.355031 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:19.408237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e0e71-a1b0-442d-ac06-c1905483a16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:19.409570 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e0e71-a1b0-442d-ac06-c1905483a16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:19.409849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e0e71-a1b0-442d-ac06-c1905483a16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:19.410124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e0e71-a1b0-442d-ac06-c1905483a16b admin admin] PATH_INFO: `/v3/users/ee3d2f037bd04f02901af78562b0ea12` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:19.411503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e0e71-a1b0-442d-ac06-c1905483a16b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee3d2f037bd04f02901af78562b0ea12)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:19.413489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e0e71-a1b0-442d-ac06-c1905483a16b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:19.463916 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c58e0e71-a1b0-442d-ac06-c1905483a16b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee3d2f037bd04f02901af78562b0ea12'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.472710 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c58e0e71-a1b0-442d-ac06-c1905483a16b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee3d2f037bd04f02901af78562b0ea12'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.473482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c58e0e71-a1b0-442d-ac06-c1905483a16b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee3d2f037bd04f02901af78562b0ea12'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.480521 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c58e0e71-a1b0-442d-ac06-c1905483a16b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee3d2f037bd04f02901af78562b0ea12'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.493540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-c58e0e71-a1b0-442d-ac06-c1905483a16b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee3d2f037bd04f02901af78562b0ea12'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.500394 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1280/2598] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:19 2022] DELETE /identity/v3/users/ee3d2f037bd04f02901af78562b0ea12 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:19.794804 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:19.796607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:19.811886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcb93b-dc36-4545-84b0-ae10b231d93e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:19.875663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:19.876617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:19.876914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:19.877180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] PATH_INFO: `/v3/projects/10b16fc4b46943c297cda34073249520` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:19.878485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b16fc4b46943c297cda34073249520)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:19.880472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:19.931360 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b16fc4b46943c297cda34073249520', 'request_id': 'req-0e411c2f-eb89-4c84-827a-de73d3a9585f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.937333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b16fc4b46943c297cda34073249520', 'request_id': 'req-0e411c2f-eb89-4c84-827a-de73d3a9585f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.939253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:19.943796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b16fc4b46943c297cda34073249520', 'request_id': 'req-0e411c2f-eb89-4c84-827a-de73d3a9585f'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.961450 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] The token cache is being invalidate because project 10b16fc4b46943c297cda34073249520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:19.962703 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.963129 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-0e411c2f-eb89-4c84-827a-de73d3a9585f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:19.965253 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1319/2599] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:19 2022] DELETE /identity/v3/projects/10b16fc4b46943c297cda34073249520 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:19.970690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba598f5-4405-4284-9a98-14e7e895b305 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:19.971175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba598f5-4405-4284-9a98-14e7e895b305 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:19.971462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba598f5-4405-4284-9a98-14e7e895b305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:19.990743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0ba598f5-4405-4284-9a98-14e7e895b305 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:19.991903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0ba598f5-4405-4284-9a98-14e7e895b305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:20.049325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1281/2600] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:20.057144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355dfa-5eb8-469b-8dbe-afd8a03a33b3 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.057455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355dfa-5eb8-469b-8dbe-afd8a03a33b3 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.057721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355dfa-5eb8-469b-8dbe-afd8a03a33b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.079092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-46355dfa-5eb8-469b-8dbe-afd8a03a33b3 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:20.080277 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-46355dfa-5eb8-469b-8dbe-afd8a03a33b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:20.087988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1320/2601] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:20.091996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba598f5-4405-4284-9a98-14e7e895b305 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.102102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba598f5-4405-4284-9a98-14e7e895b305 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.106589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebdf7a-c030-45e7-a191-ddc31be1b3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.107176 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdf7a-c030-45e7-a191-ddc31be1b3cd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.107454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdf7a-c030-45e7-a191-ddc31be1b3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.107724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebdf7a-c030-45e7-a191-ddc31be1b3cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.108359 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebdf7a-c030-45e7-a191-ddc31be1b3cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:20.108686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebdf7a-c030-45e7-a191-ddc31be1b3cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.110303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebdf7a-c030-45e7-a191-ddc31be1b3cd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.121516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1282/2602] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:40:20.125934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46355dfa-5eb8-469b-8dbe-afd8a03a33b3 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.134982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46355dfa-5eb8-469b-8dbe-afd8a03a33b3 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.139507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17cc03-6c6c-469a-b642-753f432cdc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.140192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17cc03-6c6c-469a-b642-753f432cdc63 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.140446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17cc03-6c6c-469a-b642-753f432cdc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.140718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17cc03-6c6c-469a-b642-753f432cdc63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.141871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17cc03-6c6c-469a-b642-753f432cdc63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.143953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17cc03-6c6c-469a-b642-753f432cdc63 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.173750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1321/2603] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:20 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:20.178138 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebdf7a-c030-45e7-a191-ddc31be1b3cd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.187943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebdf7a-c030-45e7-a191-ddc31be1b3cd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.202890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:20.250681 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526f134-32f6-448e-beb6-2894ccbce949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.251276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526f134-32f6-448e-beb6-2894ccbce949 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.251550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526f134-32f6-448e-beb6-2894ccbce949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.251817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526f134-32f6-448e-beb6-2894ccbce949 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.252502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526f134-32f6-448e-beb6-2894ccbce949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.254425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526f134-32f6-448e-beb6-2894ccbce949 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.274927 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1283/2604] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:20 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:20.278968 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17cc03-6c6c-469a-b642-753f432cdc63 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.289751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17cc03-6c6c-469a-b642-753f432cdc63 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.294247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a71507-bd00-439f-9d66-f32a28fd6ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.294828 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a71507-bd00-439f-9d66-f32a28fd6ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.295099 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a71507-bd00-439f-9d66-f32a28fd6ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.295365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a71507-bd00-439f-9d66-f32a28fd6ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.295994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a71507-bd00-439f-9d66-f32a28fd6ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.297592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a71507-bd00-439f-9d66-f32a28fd6ee9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.308267 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1322/2605] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:20.312194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526f134-32f6-448e-beb6-2894ccbce949 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.322917 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526f134-32f6-448e-beb6-2894ccbce949 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.329132 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07f71c-6d65-4105-95be-189ac7a2c4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.329741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07f71c-6d65-4105-95be-189ac7a2c4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.330170 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07f71c-6d65-4105-95be-189ac7a2c4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.330445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07f71c-6d65-4105-95be-189ac7a2c4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.331087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07f71c-6d65-4105-95be-189ac7a2c4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.332757 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07f71c-6d65-4105-95be-189ac7a2c4c5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.343712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1284/2606] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:20.348857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a71507-bd00-439f-9d66-f32a28fd6ee9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.358836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a71507-bd00-439f-9d66-f32a28fd6ee9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.363965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e29d9-64c6-4957-965f-f3ba57a43d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.364692 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e29d9-64c6-4957-965f-f3ba57a43d3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.364962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e29d9-64c6-4957-965f-f3ba57a43d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.365232 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e29d9-64c6-4957-965f-f3ba57a43d3a admin admin] PATH_INFO: `/v3/projects/21e4904695c848689ea9739cd560ba66/users/26252d4f60ee444d931a423e7fb7d259/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.377360 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07f71c-6d65-4105-95be-189ac7a2c4c5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.379144 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5c07f71c-6d65-4105-95be-189ac7a2c4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:20.380898 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e29d9-64c6-4957-965f-f3ba57a43d3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e4904695c848689ea9739cd560ba66, user_id=26252d4f60ee444d931a423e7fb7d259, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.388054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e29d9-64c6-4957-965f-f3ba57a43d3a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.401302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1323/2607] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:20 2022] PUT /identity/v3/projects/21e4904695c848689ea9739cd560ba66/users/26252d4f60ee444d931a423e7fb7d259/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:20.409937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c51e-7e65-468c-bdd1-429566456ca9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.410466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c51e-7e65-468c-bdd1-429566456ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.410747 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c51e-7e65-468c-bdd1-429566456ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.438499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4ba8c51e-7e65-468c-bdd1-429566456ca9 None None] MFA Rules not processed for user `26252d4f60ee444d931a423e7fb7d259`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:20.439698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4ba8c51e-7e65-468c-bdd1-429566456ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:20.441461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07f71c-6d65-4105-95be-189ac7a2c4c5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.491313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea24c6-f13c-4743-845e-c3430ceeaf72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.491914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea24c6-f13c-4743-845e-c3430ceeaf72 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.492216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea24c6-f13c-4743-845e-c3430ceeaf72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.492516 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea24c6-f13c-4743-845e-c3430ceeaf72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.494274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-29ea24c6-f13c-4743-845e-c3430ceeaf72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:20.503987 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea24c6-f13c-4743-845e-c3430ceeaf72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.505731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea24c6-f13c-4743-845e-c3430ceeaf72 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.505851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1324/2608] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:20.510124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c51e-7e65-468c-bdd1-429566456ca9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.513701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1285/2609] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:40:20.570529 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c51e-7e65-468c-bdd1-429566456ca9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.575097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15f860-c82d-43af-8be8-18883b2492da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.575700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15f860-c82d-43af-8be8-18883b2492da admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.576001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15f860-c82d-43af-8be8-18883b2492da admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.576280 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15f860-c82d-43af-8be8-18883b2492da admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.576933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15f860-c82d-43af-8be8-18883b2492da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.579412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15f860-c82d-43af-8be8-18883b2492da admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.600717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:20.609482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1325/2610] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:20 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:40:20.613289 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea24c6-f13c-4743-845e-c3430ceeaf72 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.624192 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea24c6-f13c-4743-845e-c3430ceeaf72 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.680888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:20.689550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21880fc-3313-447f-8ec9-2e8cbf835870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.690159 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21880fc-3313-447f-8ec9-2e8cbf835870 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.690435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21880fc-3313-447f-8ec9-2e8cbf835870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.690703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21880fc-3313-447f-8ec9-2e8cbf835870 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.691339 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21880fc-3313-447f-8ec9-2e8cbf835870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.693308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21880fc-3313-447f-8ec9-2e8cbf835870 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.719374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1286/2611] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:20 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:20.720679 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15f860-c82d-43af-8be8-18883b2492da admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.730727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15f860-c82d-43af-8be8-18883b2492da admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.735882 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50bff0-a160-4393-939a-fac0a2308601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.736631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50bff0-a160-4393-939a-fac0a2308601 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.736893 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50bff0-a160-4393-939a-fac0a2308601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.737175 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50bff0-a160-4393-939a-fac0a2308601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.737796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50bff0-a160-4393-939a-fac0a2308601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.739371 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50bff0-a160-4393-939a-fac0a2308601 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.754352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1326/2612] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:20.758625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21880fc-3313-447f-8ec9-2e8cbf835870 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.768875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21880fc-3313-447f-8ec9-2e8cbf835870 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.773786 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3ddb7-eab2-4e5c-b26c-42b0d66ad037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.774432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3ddb7-eab2-4e5c-b26c-42b0d66ad037 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.774709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3ddb7-eab2-4e5c-b26c-42b0d66ad037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.774981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3ddb7-eab2-4e5c-b26c-42b0d66ad037 admin admin] PATH_INFO: `/v3/projects/228f725cc50247759fcdec293d6bebb6/users/2bd9728418ce48409ee7b7d10e777449/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.789795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3ddb7-eab2-4e5c-b26c-42b0d66ad037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=228f725cc50247759fcdec293d6bebb6, user_id=2bd9728418ce48409ee7b7d10e777449, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.792505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3ddb7-eab2-4e5c-b26c-42b0d66ad037 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.805904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1287/2613] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:20 2022] PUT /identity/v3/projects/228f725cc50247759fcdec293d6bebb6/users/2bd9728418ce48409ee7b7d10e777449/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:20.809997 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50bff0-a160-4393-939a-fac0a2308601 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.820661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50bff0-a160-4393-939a-fac0a2308601 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.884228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b02d25-bed4-4d96-997b-5047e0b07f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.884870 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b02d25-bed4-4d96-997b-5047e0b07f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.885144 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b02d25-bed4-4d96-997b-5047e0b07f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.885412 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b02d25-bed4-4d96-997b-5047e0b07f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.886045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b02d25-bed4-4d96-997b-5047e0b07f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.887612 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b02d25-bed4-4d96-997b-5047e0b07f0b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.897401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1327/2614] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:20.901471 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3ddb7-eab2-4e5c-b26c-42b0d66ad037 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:20.911853 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3ddb7-eab2-4e5c-b26c-42b0d66ad037 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:20.916615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e36c10-f3ac-42d1-a6e3-26b74fd50ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:20.917263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e36c10-f3ac-42d1-a6e3-26b74fd50ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.917541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e36c10-f3ac-42d1-a6e3-26b74fd50ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.917817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e36c10-f3ac-42d1-a6e3-26b74fd50ec2 admin admin] PATH_INFO: `/v3/projects/228f725cc50247759fcdec293d6bebb6/users/2bd9728418ce48409ee7b7d10e777449/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.919738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e36c10-f3ac-42d1-a6e3-26b74fd50ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=228f725cc50247759fcdec293d6bebb6, user_id=2bd9728418ce48409ee7b7d10e777449, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:20.922975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e36c10-f3ac-42d1-a6e3-26b74fd50ec2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:20.933618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1288/2615] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:20 2022] PUT /identity/v3/projects/228f725cc50247759fcdec293d6bebb6/users/2bd9728418ce48409ee7b7d10e777449/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:20.944138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e850912-be24-4ec9-9c17-844f2704a8f4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:20.944447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e850912-be24-4ec9-9c17-844f2704a8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:20.944760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e850912-be24-4ec9-9c17-844f2704a8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:20.971467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-5e850912-be24-4ec9-9c17-844f2704a8f4 None None] MFA Rules not processed for user `2bd9728418ce48409ee7b7d10e777449`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:20.972689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5e850912-be24-4ec9-9c17-844f2704a8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:21.053674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1328/2616] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:21.065544 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e36c10-f3ac-42d1-a6e3-26b74fd50ec2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:21.067330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-83e36c10-f3ac-42d1-a6e3-26b74fd50ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:21.136813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e36c10-f3ac-42d1-a6e3-26b74fd50ec2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:21.140748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e62ea-eb8c-49bc-8e6b-9c7e309a3b4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:21.141375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e62ea-eb8c-49bc-8e6b-9c7e309a3b4a service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:21.141654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e62ea-eb8c-49bc-8e6b-9c7e309a3b4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:21.141921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e62ea-eb8c-49bc-8e6b-9c7e309a3b4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:21.151110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e62ea-eb8c-49bc-8e6b-9c7e309a3b4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:21.152574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e62ea-eb8c-49bc-8e6b-9c7e309a3b4a service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:21.163037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1289/2617] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:21.268250 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e850912-be24-4ec9-9c17-844f2704a8f4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:21.277961 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e850912-be24-4ec9-9c17-844f2704a8f4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:21.282071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355239d-af48-40b2-85c5-4d98f0a2be85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:21.282661 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355239d-af48-40b2-85c5-4d98f0a2be85 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:21.282934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355239d-af48-40b2-85c5-4d98f0a2be85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:21.283198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355239d-af48-40b2-85c5-4d98f0a2be85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:21.339795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355239d-af48-40b2-85c5-4d98f0a2be85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:21.341339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355239d-af48-40b2-85c5-4d98f0a2be85 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:21.351227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1329/2618] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:22.152000 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e62ea-eb8c-49bc-8e6b-9c7e309a3b4a service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:22.153398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-250e62ea-eb8c-49bc-8e6b-9c7e309a3b4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:22.217603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e62ea-eb8c-49bc-8e6b-9c7e309a3b4a service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:22.222062 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:22.222677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:22.222954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:22.223221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] PATH_INFO: `/v3/users/a6ef75911b594356a5c5685e7b7a2792` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:22.224327 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6ef75911b594356a5c5685e7b7a2792)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:22.226256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:22.270936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6ef75911b594356a5c5685e7b7a2792'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:22.275322 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:22.279538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6ef75911b594356a5c5685e7b7a2792'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:22.280461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6ef75911b594356a5c5685e7b7a2792'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:22.288720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6ef75911b594356a5c5685e7b7a2792'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:22.300742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6ef75911b594356a5c5685e7b7a2792'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:22.306415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1290/2619] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:22 2022] DELETE /identity/v3/users/a6ef75911b594356a5c5685e7b7a2792 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:22.310826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7231-f3cf-4eea-b756-60df8d917635 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:22.311282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7231-f3cf-4eea-b756-60df8d917635 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:22.311401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7231-f3cf-4eea-b756-60df8d917635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:22.332116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ec3e7231-f3cf-4eea-b756-60df8d917635 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:22.333841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ec3e7231-f3cf-4eea-b756-60df8d917635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:22.337921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:22.393676 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1330/2620] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:22.405297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:22.406700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:22.466459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8816381-e372-4bb0-937e-6e8b2f5e8156 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:22.470038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b02bd5-8091-4573-a410-1ecd6b0328bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:22.470639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b02bd5-8091-4573-a410-1ecd6b0328bd service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:22.470916 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b02bd5-8091-4573-a410-1ecd6b0328bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:22.471180 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b02bd5-8091-4573-a410-1ecd6b0328bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:22.480018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b02bd5-8091-4573-a410-1ecd6b0328bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:22.481479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b02bd5-8091-4573-a410-1ecd6b0328bd service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:22.491364 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1291/2621] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:22.761744 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7231-f3cf-4eea-b756-60df8d917635 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:22.763207 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ec3e7231-f3cf-4eea-b756-60df8d917635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:22.777839 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7231-f3cf-4eea-b756-60df8d917635 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:22.786426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:22.787139 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:22.787477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:22.787856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] PATH_INFO: `/v3/projects/e503204069374989bd14c21845435f88` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:22.789948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e503204069374989bd14c21845435f88)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:22.792307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:22.841305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e503204069374989bd14c21845435f88', 'request_id': 'req-7bbb783c-7209-4fab-8fe3-22af841e6cf0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:22.848309 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e503204069374989bd14c21845435f88', 'request_id': 'req-7bbb783c-7209-4fab-8fe3-22af841e6cf0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:22.854665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e503204069374989bd14c21845435f88', 'request_id': 'req-7bbb783c-7209-4fab-8fe3-22af841e6cf0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:22.870900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] The token cache is being invalidate because project e503204069374989bd14c21845435f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:22.871188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:22.871815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-7bbb783c-7209-4fab-8fe3-22af841e6cf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:22.873443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1331/2622] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:22 2022] DELETE /identity/v3/projects/e503204069374989bd14c21845435f88 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:22.878287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbf50a-fd5c-4623-b680-216ec9765521 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:22.878574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbf50a-fd5c-4623-b680-216ec9765521 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:22.878855 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbf50a-fd5c-4623-b680-216ec9765521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:22.898960 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a4dbf50a-fd5c-4623-b680-216ec9765521 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:22.900340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a4dbf50a-fd5c-4623-b680-216ec9765521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:22.963552 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1292/2623] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:22.971915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a8712-c1a0-4310-b6c0-c7f4acb39c63 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:22.972193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a8712-c1a0-4310-b6c0-c7f4acb39c63 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:22.972496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a8712-c1a0-4310-b6c0-c7f4acb39c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:22.992350 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-d35a8712-c1a0-4310-b6c0-c7f4acb39c63 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:22.993590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d35a8712-c1a0-4310-b6c0-c7f4acb39c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:23.001449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1332/2624] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:23.005615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbf50a-fd5c-4623-b680-216ec9765521 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:23.014986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbf50a-fd5c-4623-b680-216ec9765521 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:23.019971 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710960a8-5416-4ba9-83f8-03487cbf8fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:23.020601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710960a8-5416-4ba9-83f8-03487cbf8fbd admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:23.020879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710960a8-5416-4ba9-83f8-03487cbf8fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:23.021148 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710960a8-5416-4ba9-83f8-03487cbf8fbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:23.021788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710960a8-5416-4ba9-83f8-03487cbf8fbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:23.022097 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710960a8-5416-4ba9-83f8-03487cbf8fbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:23.023672 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710960a8-5416-4ba9-83f8-03487cbf8fbd admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:23.035730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1293/2625] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:40:23.040272 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a8712-c1a0-4310-b6c0-c7f4acb39c63 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:23.049219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a8712-c1a0-4310-b6c0-c7f4acb39c63 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:23.053804 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45422b7-df44-4eb6-91ae-66fc47298297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:23.054391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45422b7-df44-4eb6-91ae-66fc47298297 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:23.054654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45422b7-df44-4eb6-91ae-66fc47298297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:23.054917 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45422b7-df44-4eb6-91ae-66fc47298297 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:23.055551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45422b7-df44-4eb6-91ae-66fc47298297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:23.057467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45422b7-df44-4eb6-91ae-66fc47298297 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:23.084910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1333/2626] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:23 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:40:23.089005 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710960a8-5416-4ba9-83f8-03487cbf8fbd admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:23.099804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710960a8-5416-4ba9-83f8-03487cbf8fbd admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:23.161934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff5339-aef2-4978-ac62-f907f826a200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:23.162597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff5339-aef2-4978-ac62-f907f826a200 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:23.162818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff5339-aef2-4978-ac62-f907f826a200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:23.163087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff5339-aef2-4978-ac62-f907f826a200 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:23.163816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff5339-aef2-4978-ac62-f907f826a200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:23.165690 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff5339-aef2-4978-ac62-f907f826a200 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:23.174680 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:23.190498 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1294/2627] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:23 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:23.194961 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45422b7-df44-4eb6-91ae-66fc47298297 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:23.205591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45422b7-df44-4eb6-91ae-66fc47298297 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:23.210099 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be985b-6122-4a93-adaa-ec75bbd92fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:23.210683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be985b-6122-4a93-adaa-ec75bbd92fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:23.210954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be985b-6122-4a93-adaa-ec75bbd92fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:23.211216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be985b-6122-4a93-adaa-ec75bbd92fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:23.211865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be985b-6122-4a93-adaa-ec75bbd92fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:23.213464 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be985b-6122-4a93-adaa-ec75bbd92fdd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:23.223837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1334/2628] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:23.227745 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff5339-aef2-4978-ac62-f907f826a200 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:23.237726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff5339-aef2-4978-ac62-f907f826a200 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:23.242626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a429c-efae-42d1-834f-5f2a8cef8de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:23.243011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a429c-efae-42d1-834f-5f2a8cef8de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:23.243263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a429c-efae-42d1-834f-5f2a8cef8de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:23.243529 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a429c-efae-42d1-834f-5f2a8cef8de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:23.244187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a429c-efae-42d1-834f-5f2a8cef8de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:23.245928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a429c-efae-42d1-834f-5f2a8cef8de6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:23.256250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1295/2629] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:23.260217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be985b-6122-4a93-adaa-ec75bbd92fdd admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:23.270516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be985b-6122-4a93-adaa-ec75bbd92fdd admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:23.274985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac4a8b-81e6-4ada-812e-c06d477a57b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:23.275617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac4a8b-81e6-4ada-812e-c06d477a57b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:23.275886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac4a8b-81e6-4ada-812e-c06d477a57b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:23.276150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac4a8b-81e6-4ada-812e-c06d477a57b2 admin admin] PATH_INFO: `/v3/projects/fb58f1584369484f801e9fbfd0a7f6f9/users/7ac8dd4bf0f44c768dd9ef0ef1b6f9a6/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:23.290742 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac4a8b-81e6-4ada-812e-c06d477a57b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb58f1584369484f801e9fbfd0a7f6f9, user_id=7ac8dd4bf0f44c768dd9ef0ef1b6f9a6, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:23.293363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac4a8b-81e6-4ada-812e-c06d477a57b2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:23.305582 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1335/2630] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:23 2022] PUT /identity/v3/projects/fb58f1584369484f801e9fbfd0a7f6f9/users/7ac8dd4bf0f44c768dd9ef0ef1b6f9a6/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:23.315695 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2674d-9fca-408b-8f57-654835f3267b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:23.315981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2674d-9fca-408b-8f57-654835f3267b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:23.316250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2674d-9fca-408b-8f57-654835f3267b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:23.343427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-79b2674d-9fca-408b-8f57-654835f3267b None None] MFA Rules not processed for user `7ac8dd4bf0f44c768dd9ef0ef1b6f9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:23.345236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-79b2674d-9fca-408b-8f57-654835f3267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:23.408107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1296/2631] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:23.419031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac4a8b-81e6-4ada-812e-c06d477a57b2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:23.420721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1bac4a8b-81e6-4ada-812e-c06d477a57b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:23.451823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:23.478243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac4a8b-81e6-4ada-812e-c06d477a57b2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:23.481883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408f701-72d3-4eb8-82e7-b896eaee5958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:23.482479 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408f701-72d3-4eb8-82e7-b896eaee5958 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:23.482757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408f701-72d3-4eb8-82e7-b896eaee5958 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:23.483024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408f701-72d3-4eb8-82e7-b896eaee5958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:23.491101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408f701-72d3-4eb8-82e7-b896eaee5958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:23.492537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408f701-72d3-4eb8-82e7-b896eaee5958 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:23.501517 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1336/2632] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:23.958276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2674d-9fca-408b-8f57-654835f3267b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:23.959740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-79b2674d-9fca-408b-8f57-654835f3267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:24.021355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2674d-9fca-408b-8f57-654835f3267b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:24.025824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:24.026440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:24.026715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:24.026984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] PATH_INFO: `/v3/users/26252d4f60ee444d931a423e7fb7d259` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:24.028070 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26252d4f60ee444d931a423e7fb7d259)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:24.030011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:24.076645 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26252d4f60ee444d931a423e7fb7d259'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.083357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26252d4f60ee444d931a423e7fb7d259'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.084010 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26252d4f60ee444d931a423e7fb7d259'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.091192 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26252d4f60ee444d931a423e7fb7d259'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.096337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:24.104315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26252d4f60ee444d931a423e7fb7d259'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.110838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1297/2633] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:23 2022] DELETE /identity/v3/users/26252d4f60ee444d931a423e7fb7d259 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:24.115259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125065d-7426-4b46-af76-93cccfb13793 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:24.115532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125065d-7426-4b46-af76-93cccfb13793 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:24.115803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125065d-7426-4b46-af76-93cccfb13793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:24.135282 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7125065d-7426-4b46-af76-93cccfb13793 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:24.136592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7125065d-7426-4b46-af76-93cccfb13793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:24.193921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1337/2634] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:24.203559 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:24.204977 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:24.259921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078d08c-056e-4e75-a8ea-64f1a1fbc663 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:24.263452 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce8693-81f8-46ed-870f-50e8dcffec8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:24.264040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce8693-81f8-46ed-870f-50e8dcffec8d service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:24.264310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce8693-81f8-46ed-870f-50e8dcffec8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:24.264611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce8693-81f8-46ed-870f-50e8dcffec8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:24.273338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce8693-81f8-46ed-870f-50e8dcffec8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:24.274764 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce8693-81f8-46ed-870f-50e8dcffec8d service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:24.284303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1298/2635] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:24.550187 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125065d-7426-4b46-af76-93cccfb13793 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:24.551684 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7125065d-7426-4b46-af76-93cccfb13793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:24.564931 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125065d-7426-4b46-af76-93cccfb13793 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:24.569765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:24.570400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:24.570675 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:24.570943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] PATH_INFO: `/v3/projects/21e4904695c848689ea9739cd560ba66` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:24.572044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e4904695c848689ea9739cd560ba66)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:24.573971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:24.619708 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e4904695c848689ea9739cd560ba66', 'request_id': 'req-d2291c04-9aca-40b1-b39a-20467dc31c9d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.625833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e4904695c848689ea9739cd560ba66', 'request_id': 'req-d2291c04-9aca-40b1-b39a-20467dc31c9d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.631788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e4904695c848689ea9739cd560ba66', 'request_id': 'req-d2291c04-9aca-40b1-b39a-20467dc31c9d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.643729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] The token cache is being invalidate because project 21e4904695c848689ea9739cd560ba66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:24.644011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.644649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d2291c04-9aca-40b1-b39a-20467dc31c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.646141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1338/2636] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:24 2022] DELETE /identity/v3/projects/21e4904695c848689ea9739cd560ba66 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:24.650073 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce8693-81f8-46ed-870f-50e8dcffec8d service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:24.659631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce8693-81f8-46ed-870f-50e8dcffec8d service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:24.714520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:24.715134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:24.715415 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:24.715731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] PATH_INFO: `/v3/users/2bd9728418ce48409ee7b7d10e777449` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:24.716864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd9728418ce48409ee7b7d10e777449)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:24.718780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:24.764257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd9728418ce48409ee7b7d10e777449'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.771101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd9728418ce48409ee7b7d10e777449'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.771744 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd9728418ce48409ee7b7d10e777449'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.779035 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd9728418ce48409ee7b7d10e777449'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.792537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd9728418ce48409ee7b7d10e777449'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:24.799124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1299/2637] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:24 2022] DELETE /identity/v3/users/2bd9728418ce48409ee7b7d10e777449 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:24.856553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b082b-5fdf-4f14-8224-0c84aad419f6 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:24.856887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b082b-5fdf-4f14-8224-0c84aad419f6 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:24.857133 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b082b-5fdf-4f14-8224-0c84aad419f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:24.878605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-678b082b-5fdf-4f14-8224-0c84aad419f6 None None] MFA Rules not processed for user `7ac8dd4bf0f44c768dd9ef0ef1b6f9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:24.879809 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-678b082b-5fdf-4f14-8224-0c84aad419f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:24.929140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:24.938135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1339/2638] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:25.073956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:25.075372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:25.086880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:25.143597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2ca1f-716c-4606-8605-18f3c956a498 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:25.148215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:25.148873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.149152 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.149422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] PATH_INFO: `/v3/projects/228f725cc50247759fcdec293d6bebb6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.150566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=228f725cc50247759fcdec293d6bebb6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:25.152497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:25.203340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '228f725cc50247759fcdec293d6bebb6', 'request_id': 'req-149ef258-f05b-4917-8c7f-1a17e7cc8768'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:25.209273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '228f725cc50247759fcdec293d6bebb6', 'request_id': 'req-149ef258-f05b-4917-8c7f-1a17e7cc8768'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:25.215522 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '228f725cc50247759fcdec293d6bebb6', 'request_id': 'req-149ef258-f05b-4917-8c7f-1a17e7cc8768'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:25.228569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] The token cache is being invalidate because project 228f725cc50247759fcdec293d6bebb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:25.228859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:25.229670 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-149ef258-f05b-4917-8c7f-1a17e7cc8768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:25.231193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1300/2639] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:25 2022] DELETE /identity/v3/projects/228f725cc50247759fcdec293d6bebb6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:25.236810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d13227-acac-4673-9e39-c665538af919 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.236937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d13227-acac-4673-9e39-c665538af919 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.237130 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d13227-acac-4673-9e39-c665538af919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.257565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-19d13227-acac-4673-9e39-c665538af919 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:25.258747 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-19d13227-acac-4673-9e39-c665538af919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:25.317081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1340/2640] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:25.324937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ad7fc-a5aa-4f3a-844b-c8e4c6348adb None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.325221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ad7fc-a5aa-4f3a-844b-c8e4c6348adb None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.325490 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ad7fc-a5aa-4f3a-844b-c8e4c6348adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.345985 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-247ad7fc-a5aa-4f3a-844b-c8e4c6348adb None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:25.347280 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-247ad7fc-a5aa-4f3a-844b-c8e4c6348adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:25.355371 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1301/2641] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:25.359608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d13227-acac-4673-9e39-c665538af919 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:25.369483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d13227-acac-4673-9e39-c665538af919 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:25.374982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d6654-1ab6-4768-80d8-616144cd71be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:25.375611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d6654-1ab6-4768-80d8-616144cd71be admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.375887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d6654-1ab6-4768-80d8-616144cd71be admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.376149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d6654-1ab6-4768-80d8-616144cd71be admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.376818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d6654-1ab6-4768-80d8-616144cd71be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:25.377255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d6654-1ab6-4768-80d8-616144cd71be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:25.378867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d6654-1ab6-4768-80d8-616144cd71be admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:25.394364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1341/2642] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:40:25.398643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ad7fc-a5aa-4f3a-844b-c8e4c6348adb None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:25.408677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ad7fc-a5aa-4f3a-844b-c8e4c6348adb None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:25.416127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afea6a-2166-4802-89a5-fc96d67c93e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:25.416796 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afea6a-2166-4802-89a5-fc96d67c93e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.417075 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afea6a-2166-4802-89a5-fc96d67c93e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.417346 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afea6a-2166-4802-89a5-fc96d67c93e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.418116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afea6a-2166-4802-89a5-fc96d67c93e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:25.420118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afea6a-2166-4802-89a5-fc96d67c93e8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:25.431886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d6654-1ab6-4768-80d8-616144cd71be admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:25.433759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8e0d6654-1ab6-4768-80d8-616144cd71be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:25.461017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1302/2643] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:25 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:25.461783 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afea6a-2166-4802-89a5-fc96d67c93e8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:25.514333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d6654-1ab6-4768-80d8-616144cd71be admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:25.529459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afea6a-2166-4802-89a5-fc96d67c93e8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:25.534991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d49d2c-a7f4-4c9e-8ee1-7fbbc2afc6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:25.535856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d49d2c-a7f4-4c9e-8ee1-7fbbc2afc6ca admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.536134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d49d2c-a7f4-4c9e-8ee1-7fbbc2afc6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.536427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d49d2c-a7f4-4c9e-8ee1-7fbbc2afc6ca admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.537049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d49d2c-a7f4-4c9e-8ee1-7fbbc2afc6ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:25.539037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d49d2c-a7f4-4c9e-8ee1-7fbbc2afc6ca admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:25.556214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:25.564769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1303/2644] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:25 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:25.566089 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d49d2c-a7f4-4c9e-8ee1-7fbbc2afc6ca admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:25.575526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54757dd4-bd24-49fc-ad06-2f1bb891a0fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:25.576077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d49d2c-a7f4-4c9e-8ee1-7fbbc2afc6ca admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:25.576186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54757dd4-bd24-49fc-ad06-2f1bb891a0fa service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.576420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54757dd4-bd24-49fc-ad06-2f1bb891a0fa service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.576741 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54757dd4-bd24-49fc-ad06-2f1bb891a0fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.579073 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-54757dd4-bd24-49fc-ad06-2f1bb891a0fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:25.580857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed578ada-f578-40ad-bc55-38ebbce95b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:25.581433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed578ada-f578-40ad-bc55-38ebbce95b8b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.581711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed578ada-f578-40ad-bc55-38ebbce95b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.581982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed578ada-f578-40ad-bc55-38ebbce95b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.582619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed578ada-f578-40ad-bc55-38ebbce95b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:25.584187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed578ada-f578-40ad-bc55-38ebbce95b8b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:25.590563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54757dd4-bd24-49fc-ad06-2f1bb891a0fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:25.592112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54757dd4-bd24-49fc-ad06-2f1bb891a0fa service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:25.594588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1304/2645] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:40:25.598592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed578ada-f578-40ad-bc55-38ebbce95b8b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:25.602819 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1342/2646] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:40:25.608012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed578ada-f578-40ad-bc55-38ebbce95b8b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:25.612577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902a3d7-6b2d-4dc3-82cf-01557e2c0281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:25.613148 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a3d7-6b2d-4dc3-82cf-01557e2c0281 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.613426 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a3d7-6b2d-4dc3-82cf-01557e2c0281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.613712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a3d7-6b2d-4dc3-82cf-01557e2c0281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.614329 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902a3d7-6b2d-4dc3-82cf-01557e2c0281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:25.615880 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902a3d7-6b2d-4dc3-82cf-01557e2c0281 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:25.626694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1305/2647] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:40:25.630823 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54757dd4-bd24-49fc-ad06-2f1bb891a0fa service placement] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:25.641594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54757dd4-bd24-49fc-ad06-2f1bb891a0fa service placement] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:25.646353 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01ea51-2124-4e4e-9265-334ed6135803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:25.647001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ea51-2124-4e4e-9265-334ed6135803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.647318 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ea51-2124-4e4e-9265-334ed6135803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.647591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ea51-2124-4e4e-9265-334ed6135803 admin admin] PATH_INFO: `/v3/projects/406caa137cfb41d0832dd8cebc5dcd9a/users/1a280c3950f84b15a4442551bef9839e/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.664539 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01ea51-2124-4e4e-9265-334ed6135803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=406caa137cfb41d0832dd8cebc5dcd9a, user_id=1a280c3950f84b15a4442551bef9839e, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:25.667155 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01ea51-2124-4e4e-9265-334ed6135803 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:25.670093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:25.682373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1343/2648] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:25 2022] PUT /identity/v3/projects/406caa137cfb41d0832dd8cebc5dcd9a/users/1a280c3950f84b15a4442551bef9839e/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:25.694215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e6579-8ece-41ed-b41e-ab57b511dd98 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.694721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e6579-8ece-41ed-b41e-ab57b511dd98 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.695012 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e6579-8ece-41ed-b41e-ab57b511dd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.722545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-084e6579-8ece-41ed-b41e-ab57b511dd98 None None] MFA Rules not processed for user `1a280c3950f84b15a4442551bef9839e`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:25.723724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-084e6579-8ece-41ed-b41e-ab57b511dd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:25.799648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1306/2649] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:25.804098 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01ea51-2124-4e4e-9265-334ed6135803 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:25.870395 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01ea51-2124-4e4e-9265-334ed6135803 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:25.876654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43c94b-7fd9-4614-b77c-5202fb116b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:25.877368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43c94b-7fd9-4614-b77c-5202fb116b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.877704 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43c94b-7fd9-4614-b77c-5202fb116b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.878119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43c94b-7fd9-4614-b77c-5202fb116b4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.879016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43c94b-7fd9-4614-b77c-5202fb116b4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:25.881323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43c94b-7fd9-4614-b77c-5202fb116b4e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:25.914683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1344/2650] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:25 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:40:25.918542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e6579-8ece-41ed-b41e-ab57b511dd98 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:25.929823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e6579-8ece-41ed-b41e-ab57b511dd98 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:25.991110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c930493-2eeb-4fca-9ccb-55b2d0f5dc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:25.991711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c930493-2eeb-4fca-9ccb-55b2d0f5dc48 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:25.991988 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c930493-2eeb-4fca-9ccb-55b2d0f5dc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:25.992254 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c930493-2eeb-4fca-9ccb-55b2d0f5dc48 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:25.992915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c930493-2eeb-4fca-9ccb-55b2d0f5dc48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:25.994854 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c930493-2eeb-4fca-9ccb-55b2d0f5dc48 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:26.015505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1307/2651] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:25 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:40:26.019611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43c94b-7fd9-4614-b77c-5202fb116b4e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:26.031654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43c94b-7fd9-4614-b77c-5202fb116b4e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:26.036913 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5188c00-8f68-4c7b-a337-e4bff7625dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:26.037522 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5188c00-8f68-4c7b-a337-e4bff7625dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:26.037795 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5188c00-8f68-4c7b-a337-e4bff7625dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:26.038065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5188c00-8f68-4c7b-a337-e4bff7625dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:26.038852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5188c00-8f68-4c7b-a337-e4bff7625dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:26.040618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5188c00-8f68-4c7b-a337-e4bff7625dbd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:26.055405 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1345/2652] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:26.056465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c930493-2eeb-4fca-9ccb-55b2d0f5dc48 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:26.067617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c930493-2eeb-4fca-9ccb-55b2d0f5dc48 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:26.073799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fed90-743c-41a6-9143-3d1fb88a9eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:26.074432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fed90-743c-41a6-9143-3d1fb88a9eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:26.074707 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fed90-743c-41a6-9143-3d1fb88a9eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:26.074972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fed90-743c-41a6-9143-3d1fb88a9eaa admin admin] PATH_INFO: `/v3/projects/106451ed99b14ba5b87069de790223a3/users/99e7f25af11d4f27b2f679bc8dd182f4/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:26.091324 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fed90-743c-41a6-9143-3d1fb88a9eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=106451ed99b14ba5b87069de790223a3, user_id=99e7f25af11d4f27b2f679bc8dd182f4, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:26.094277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fed90-743c-41a6-9143-3d1fb88a9eaa admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:26.108111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1308/2653] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:26 2022] PUT /identity/v3/projects/106451ed99b14ba5b87069de790223a3/users/99e7f25af11d4f27b2f679bc8dd182f4/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:26.113001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5188c00-8f68-4c7b-a337-e4bff7625dbd admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:26.124408 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5188c00-8f68-4c7b-a337-e4bff7625dbd admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:26.181841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2d43b-17e0-4bf5-ad88-ad72f71b9136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:26.182439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2d43b-17e0-4bf5-ad88-ad72f71b9136 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:26.182711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2d43b-17e0-4bf5-ad88-ad72f71b9136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:26.182975 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2d43b-17e0-4bf5-ad88-ad72f71b9136 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:26.183634 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2d43b-17e0-4bf5-ad88-ad72f71b9136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:26.185233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2d43b-17e0-4bf5-ad88-ad72f71b9136 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:26.195814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1346/2654] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:26.200043 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fed90-743c-41a6-9143-3d1fb88a9eaa admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:26.210343 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fed90-743c-41a6-9143-3d1fb88a9eaa admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:26.216136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5431908-2a17-4a9e-b2cd-18931515259f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:26.216818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431908-2a17-4a9e-b2cd-18931515259f admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:26.217223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431908-2a17-4a9e-b2cd-18931515259f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:26.217357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431908-2a17-4a9e-b2cd-18931515259f admin admin] PATH_INFO: `/v3/projects/106451ed99b14ba5b87069de790223a3/users/99e7f25af11d4f27b2f679bc8dd182f4/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:26.219540 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5431908-2a17-4a9e-b2cd-18931515259f admin admin] RBAC: Authorizing `identity:create_grant(project_id=106451ed99b14ba5b87069de790223a3, user_id=99e7f25af11d4f27b2f679bc8dd182f4, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:26.222223 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5431908-2a17-4a9e-b2cd-18931515259f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:26.233727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1309/2655] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:26 2022] PUT /identity/v3/projects/106451ed99b14ba5b87069de790223a3/users/99e7f25af11d4f27b2f679bc8dd182f4/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:26.244521 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d995f0-c915-47ca-86e9-ad5d0b012173 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:26.244807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d995f0-c915-47ca-86e9-ad5d0b012173 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:26.245119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d995f0-c915-47ca-86e9-ad5d0b012173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:26.275267 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-05d995f0-c915-47ca-86e9-ad5d0b012173 None None] MFA Rules not processed for user `99e7f25af11d4f27b2f679bc8dd182f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:26.276520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-05d995f0-c915-47ca-86e9-ad5d0b012173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:26.327020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:26.345504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1347/2656] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:26.356603 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5431908-2a17-4a9e-b2cd-18931515259f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:26.358648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a5431908-2a17-4a9e-b2cd-18931515259f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:26.382157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:26.418292 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5431908-2a17-4a9e-b2cd-18931515259f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:26.421891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2628-6b9d-4a4c-9de4-ec7a4f1ccbbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:26.423045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2628-6b9d-4a4c-9de4-ec7a4f1ccbbd service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:26.423319 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2628-6b9d-4a4c-9de4-ec7a4f1ccbbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:26.423585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2628-6b9d-4a4c-9de4-ec7a4f1ccbbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:26.432305 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2628-6b9d-4a4c-9de4-ec7a4f1ccbbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:26.433766 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2628-6b9d-4a4c-9de4-ec7a4f1ccbbd service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:26.442944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1310/2657] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:26.524054 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d995f0-c915-47ca-86e9-ad5d0b012173 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:26.534150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d995f0-c915-47ca-86e9-ad5d0b012173 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:26.538172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3250d-413a-4cba-afd2-fef5d544fc4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:26.538765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3250d-413a-4cba-afd2-fef5d544fc4b service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:26.539039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3250d-413a-4cba-afd2-fef5d544fc4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:26.539306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3250d-413a-4cba-afd2-fef5d544fc4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:26.598152 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3250d-413a-4cba-afd2-fef5d544fc4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:26.599600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3250d-413a-4cba-afd2-fef5d544fc4b service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:26.610396 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1348/2658] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:31.418967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2628-6b9d-4a4c-9de4-ec7a4f1ccbbd service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:31.480912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2628-6b9d-4a4c-9de4-ec7a4f1ccbbd service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:31.485610 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:31.486234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:31.486514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:31.486789 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] PATH_INFO: `/v3/users/1a280c3950f84b15a4442551bef9839e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:31.487904 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a280c3950f84b15a4442551bef9839e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:31.489895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:31.536573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a280c3950f84b15a4442551bef9839e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:31.543434 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a280c3950f84b15a4442551bef9839e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:31.544102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a280c3950f84b15a4442551bef9839e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:31.550577 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a280c3950f84b15a4442551bef9839e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:31.562528 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a280c3950f84b15a4442551bef9839e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:31.568392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1311/2659] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:31 2022] DELETE /identity/v3/users/1a280c3950f84b15a4442551bef9839e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:31.572970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b63a-e4ec-4774-8db0-a026d2a12267 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:31.573434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b63a-e4ec-4774-8db0-a026d2a12267 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:31.573551 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b63a-e4ec-4774-8db0-a026d2a12267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:31.592690 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b578b63a-e4ec-4774-8db0-a026d2a12267 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:31.594496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b578b63a-e4ec-4774-8db0-a026d2a12267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:31.653445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1349/2660] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:31.663476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:31.664910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:31.713046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:31.727640 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcacf8-ebc2-4e82-bee5-acc8584a50d8 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:31.731281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2a3af-66df-42f5-bb5b-4aaa9364e5b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:31.731818 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2a3af-66df-42f5-bb5b-4aaa9364e5b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:31.732090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2a3af-66df-42f5-bb5b-4aaa9364e5b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:31.732356 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2a3af-66df-42f5-bb5b-4aaa9364e5b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:31.742116 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2a3af-66df-42f5-bb5b-4aaa9364e5b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:31.743535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2a3af-66df-42f5-bb5b-4aaa9364e5b9 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:31.753758 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1312/2661] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:32.020056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578b63a-e4ec-4774-8db0-a026d2a12267 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:32.021590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b578b63a-e4ec-4774-8db0-a026d2a12267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:32.034401 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578b63a-e4ec-4774-8db0-a026d2a12267 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:32.039322 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:32.039951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:32.040229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:32.040541 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] PATH_INFO: `/v3/projects/406caa137cfb41d0832dd8cebc5dcd9a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:32.041655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=406caa137cfb41d0832dd8cebc5dcd9a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:32.043530 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:32.089106 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '406caa137cfb41d0832dd8cebc5dcd9a', 'request_id': 'req-d70c37a9-195f-4b15-be38-52b4ee5eddc5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.094635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '406caa137cfb41d0832dd8cebc5dcd9a', 'request_id': 'req-d70c37a9-195f-4b15-be38-52b4ee5eddc5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.100633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '406caa137cfb41d0832dd8cebc5dcd9a', 'request_id': 'req-d70c37a9-195f-4b15-be38-52b4ee5eddc5'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.113021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] The token cache is being invalidate because project 406caa137cfb41d0832dd8cebc5dcd9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:32.113301 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.113915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.115354 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1350/2662] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:32 2022] DELETE /identity/v3/projects/406caa137cfb41d0832dd8cebc5dcd9a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:32.119253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2a3af-66df-42f5-bb5b-4aaa9364e5b9 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:32.129130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2a3af-66df-42f5-bb5b-4aaa9364e5b9 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:32.184271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a9fcd-cac5-4da1-a3db-0a343bdf387a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:32.184915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a9fcd-cac5-4da1-a3db-0a343bdf387a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:32.185192 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a9fcd-cac5-4da1-a3db-0a343bdf387a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:32.185463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a9fcd-cac5-4da1-a3db-0a343bdf387a admin admin] PATH_INFO: `/v3/users/99e7f25af11d4f27b2f679bc8dd182f4` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:32.186569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a9fcd-cac5-4da1-a3db-0a343bdf387a admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e7f25af11d4f27b2f679bc8dd182f4)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:32.189113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a9fcd-cac5-4da1-a3db-0a343bdf387a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:32.233866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eb8a9fcd-cac5-4da1-a3db-0a343bdf387a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e7f25af11d4f27b2f679bc8dd182f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.241119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eb8a9fcd-cac5-4da1-a3db-0a343bdf387a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e7f25af11d4f27b2f679bc8dd182f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.241777 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eb8a9fcd-cac5-4da1-a3db-0a343bdf387a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e7f25af11d4f27b2f679bc8dd182f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.248951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eb8a9fcd-cac5-4da1-a3db-0a343bdf387a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e7f25af11d4f27b2f679bc8dd182f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.261190 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-eb8a9fcd-cac5-4da1-a3db-0a343bdf387a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e7f25af11d4f27b2f679bc8dd182f4'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.267040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1313/2663] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:32 2022] DELETE /identity/v3/users/99e7f25af11d4f27b2f679bc8dd182f4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:32.610227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:32.611700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:32.624681 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c37a9-195f-4b15-be38-52b4ee5eddc5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:32.634305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:32.684697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:32.685321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:32.685591 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:32.685861 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] PATH_INFO: `/v3/projects/106451ed99b14ba5b87069de790223a3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:32.686950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=106451ed99b14ba5b87069de790223a3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:32.688901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:32.733211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '106451ed99b14ba5b87069de790223a3', 'request_id': 'req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.739404 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '106451ed99b14ba5b87069de790223a3', 'request_id': 'req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.745304 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '106451ed99b14ba5b87069de790223a3', 'request_id': 'req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.757976 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] The token cache is being invalidate because project 106451ed99b14ba5b87069de790223a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:32.758259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.758874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d5b1160f-b7e0-4aec-a7c8-13bb88ea88b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:32.760331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1351/2664] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:32 2022] DELETE /identity/v3/projects/106451ed99b14ba5b87069de790223a3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:32.765149 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0db181-cc6d-4ae9-94e6-3b06d6dd6e1b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:32.766137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0db181-cc6d-4ae9-94e6-3b06d6dd6e1b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:32.766239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0db181-cc6d-4ae9-94e6-3b06d6dd6e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:32.786127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3e0db181-cc6d-4ae9-94e6-3b06d6dd6e1b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:32.787312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3e0db181-cc6d-4ae9-94e6-3b06d6dd6e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:32.843062 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1314/2665] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:32.852656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc826-3d40-450a-850b-3efbf90288ea None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:32.852937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc826-3d40-450a-850b-3efbf90288ea None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:32.853206 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc826-3d40-450a-850b-3efbf90288ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:32.872782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0c1dc826-3d40-450a-850b-3efbf90288ea None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:32.873962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0c1dc826-3d40-450a-850b-3efbf90288ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:32.881594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1352/2666] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:32.885759 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0db181-cc6d-4ae9-94e6-3b06d6dd6e1b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:32.892834 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:32.896857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0db181-cc6d-4ae9-94e6-3b06d6dd6e1b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:32.902895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95ba6e-bf86-4eb8-8750-938c136ad598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:32.903832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95ba6e-bf86-4eb8-8750-938c136ad598 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:32.904278 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95ba6e-bf86-4eb8-8750-938c136ad598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:32.904635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95ba6e-bf86-4eb8-8750-938c136ad598 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:32.905438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95ba6e-bf86-4eb8-8750-938c136ad598 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:32.905737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95ba6e-bf86-4eb8-8750-938c136ad598 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:32.907956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95ba6e-bf86-4eb8-8750-938c136ad598 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:32.919632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1315/2667] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:40:32.923862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dc826-3d40-450a-850b-3efbf90288ea None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:32.932864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dc826-3d40-450a-850b-3efbf90288ea None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:32.937387 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfdda1-e2ba-4e41-8142-5beee938b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:32.937971 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfdda1-e2ba-4e41-8142-5beee938b011 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:32.938241 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfdda1-e2ba-4e41-8142-5beee938b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:32.938505 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfdda1-e2ba-4e41-8142-5beee938b011 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:32.939116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfdda1-e2ba-4e41-8142-5beee938b011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:32.941018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfdda1-e2ba-4e41-8142-5beee938b011 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:32.967071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1353/2668] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:32 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:32.971699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95ba6e-bf86-4eb8-8750-938c136ad598 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:32.981461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95ba6e-bf86-4eb8-8750-938c136ad598 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.035236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d3c48-dc86-4027-aa5c-650e96d636b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.036083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d3c48-dc86-4027-aa5c-650e96d636b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.036361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d3c48-dc86-4027-aa5c-650e96d636b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.036654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d3c48-dc86-4027-aa5c-650e96d636b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.037257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d3c48-dc86-4027-aa5c-650e96d636b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.039147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d3c48-dc86-4027-aa5c-650e96d636b5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.059405 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1316/2669] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:32 2022] POST /identity/v3/users => generated 288 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:33.063467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfdda1-e2ba-4e41-8142-5beee938b011 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.073614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfdda1-e2ba-4e41-8142-5beee938b011 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.078723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842136b8-5ff0-4c20-9dfb-81ec23e6dbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.079316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842136b8-5ff0-4c20-9dfb-81ec23e6dbac admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.079592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842136b8-5ff0-4c20-9dfb-81ec23e6dbac admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.079854 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842136b8-5ff0-4c20-9dfb-81ec23e6dbac admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.080524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842136b8-5ff0-4c20-9dfb-81ec23e6dbac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.082092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842136b8-5ff0-4c20-9dfb-81ec23e6dbac admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.091953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1354/2670] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:40:33.096427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d3c48-dc86-4027-aa5c-650e96d636b5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.106252 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d3c48-dc86-4027-aa5c-650e96d636b5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.111701 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923f35d-41ab-43c8-bde4-94f8e2bd6c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.111863 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923f35d-41ab-43c8-bde4-94f8e2bd6c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.112080 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923f35d-41ab-43c8-bde4-94f8e2bd6c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.112505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923f35d-41ab-43c8-bde4-94f8e2bd6c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.113157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923f35d-41ab-43c8-bde4-94f8e2bd6c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.115009 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923f35d-41ab-43c8-bde4-94f8e2bd6c2b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.127867 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1317/2671] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:33.132007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842136b8-5ff0-4c20-9dfb-81ec23e6dbac admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.142733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842136b8-5ff0-4c20-9dfb-81ec23e6dbac admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.147604 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcb5b9-670c-4847-b5a8-bc1e073365f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.148278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcb5b9-670c-4847-b5a8-bc1e073365f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.148559 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcb5b9-670c-4847-b5a8-bc1e073365f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.148833 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcb5b9-670c-4847-b5a8-bc1e073365f1 admin admin] PATH_INFO: `/v3/projects/b85f707c20d14e8c8f71c1eccd6c6d87/users/3e1f2741769346d6aa664815474ec749/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.164541 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcb5b9-670c-4847-b5a8-bc1e073365f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b85f707c20d14e8c8f71c1eccd6c6d87, user_id=3e1f2741769346d6aa664815474ec749, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.167175 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcb5b9-670c-4847-b5a8-bc1e073365f1 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.181810 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1355/2672] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:33 2022] PUT /identity/v3/projects/b85f707c20d14e8c8f71c1eccd6c6d87/users/3e1f2741769346d6aa664815474ec749/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:33.192791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c466b0-69c5-4398-9a19-4882a2661dc9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.193321 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c466b0-69c5-4398-9a19-4882a2661dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.193446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c466b0-69c5-4398-9a19-4882a2661dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.224656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b6c466b0-69c5-4398-9a19-4882a2661dc9 None None] MFA Rules not processed for user `3e1f2741769346d6aa664815474ec749`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:33.225882 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b6c466b0-69c5-4398-9a19-4882a2661dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:33.251340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcb5b9-670c-4847-b5a8-bc1e073365f1 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.253137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-c2dcb5b9-670c-4847-b5a8-bc1e073365f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:33.298308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:33.301264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1318/2673] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:33.305635 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c466b0-69c5-4398-9a19-4882a2661dc9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.334689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcb5b9-670c-4847-b5a8-bc1e073365f1 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.340784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fb802-c175-41e3-8543-35654f57dc68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.341722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fb802-c175-41e3-8543-35654f57dc68 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.341997 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fb802-c175-41e3-8543-35654f57dc68 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.343088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fb802-c175-41e3-8543-35654f57dc68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.344752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a66fb802-c175-41e3-8543-35654f57dc68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:33.357737 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fb802-c175-41e3-8543-35654f57dc68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.359254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fb802-c175-41e3-8543-35654f57dc68 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.376571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1356/2674] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:33.382319 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c466b0-69c5-4398-9a19-4882a2661dc9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.389461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be7532-2ac0-4c1c-a451-37fe9db1d707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.390222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be7532-2ac0-4c1c-a451-37fe9db1d707 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.390526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be7532-2ac0-4c1c-a451-37fe9db1d707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.391323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be7532-2ac0-4c1c-a451-37fe9db1d707 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.391956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be7532-2ac0-4c1c-a451-37fe9db1d707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.394771 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be7532-2ac0-4c1c-a451-37fe9db1d707 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.399887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:33.430673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1319/2675] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:33 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:33.434656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fb802-c175-41e3-8543-35654f57dc68 service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.445324 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fb802-c175-41e3-8543-35654f57dc68 service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.504157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad6cae-b5c6-4cf1-a380-82331c49e50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.504794 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad6cae-b5c6-4cf1-a380-82331c49e50e admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.505073 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad6cae-b5c6-4cf1-a380-82331c49e50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.505339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad6cae-b5c6-4cf1-a380-82331c49e50e admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.505968 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad6cae-b5c6-4cf1-a380-82331c49e50e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.507853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad6cae-b5c6-4cf1-a380-82331c49e50e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.530161 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1357/2676] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:33 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:33.534184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be7532-2ac0-4c1c-a451-37fe9db1d707 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.545601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be7532-2ac0-4c1c-a451-37fe9db1d707 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.552124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead2a85-b089-4bd0-8ea3-c15794937d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.552750 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead2a85-b089-4bd0-8ea3-c15794937d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.553025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead2a85-b089-4bd0-8ea3-c15794937d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.553291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead2a85-b089-4bd0-8ea3-c15794937d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.554143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead2a85-b089-4bd0-8ea3-c15794937d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.555729 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead2a85-b089-4bd0-8ea3-c15794937d17 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.566910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1320/2677] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:33.570746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad6cae-b5c6-4cf1-a380-82331c49e50e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.580906 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad6cae-b5c6-4cf1-a380-82331c49e50e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.585564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc705b1-7b58-43f5-9d6b-b92854af2c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.586158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc705b1-7b58-43f5-9d6b-b92854af2c50 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.586435 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc705b1-7b58-43f5-9d6b-b92854af2c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.586706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc705b1-7b58-43f5-9d6b-b92854af2c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.587337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc705b1-7b58-43f5-9d6b-b92854af2c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.588963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc705b1-7b58-43f5-9d6b-b92854af2c50 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.599047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1358/2678] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:33.603160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead2a85-b089-4bd0-8ea3-c15794937d17 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.613733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead2a85-b089-4bd0-8ea3-c15794937d17 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.618955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30231c7-02d0-4a3e-bdaf-7850a427b180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.619704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30231c7-02d0-4a3e-bdaf-7850a427b180 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.619952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30231c7-02d0-4a3e-bdaf-7850a427b180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.620126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30231c7-02d0-4a3e-bdaf-7850a427b180 admin admin] PATH_INFO: `/v3/projects/961152926f50440b80223ae7f1417669/users/81fc09dae84148fcbb42760d115ac870/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.635887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30231c7-02d0-4a3e-bdaf-7850a427b180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=961152926f50440b80223ae7f1417669, user_id=81fc09dae84148fcbb42760d115ac870, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.638717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30231c7-02d0-4a3e-bdaf-7850a427b180 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.653082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1321/2679] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:33 2022] PUT /identity/v3/projects/961152926f50440b80223ae7f1417669/users/81fc09dae84148fcbb42760d115ac870/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:33.663608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940f230-0ed7-4347-83d0-936e8aa35cfe None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.664168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940f230-0ed7-4347-83d0-936e8aa35cfe None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.664168 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940f230-0ed7-4347-83d0-936e8aa35cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.695140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b940f230-0ed7-4347-83d0-936e8aa35cfe None None] MFA Rules not processed for user `81fc09dae84148fcbb42760d115ac870`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:33.696444 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b940f230-0ed7-4347-83d0-936e8aa35cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:33.761950 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1359/2680] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:33.766397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30231c7-02d0-4a3e-bdaf-7850a427b180 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.827357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30231c7-02d0-4a3e-bdaf-7850a427b180 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.831913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b82261-8405-416a-a26a-13b448d3f8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.832536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82261-8405-416a-a26a-13b448d3f8c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.832817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82261-8405-416a-a26a-13b448d3f8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.833084 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82261-8405-416a-a26a-13b448d3f8c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.833704 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b82261-8405-416a-a26a-13b448d3f8c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.835578 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b82261-8405-416a-a26a-13b448d3f8c6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.861088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1322/2681] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:33 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:33.865108 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940f230-0ed7-4347-83d0-936e8aa35cfe None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.876258 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940f230-0ed7-4347-83d0-936e8aa35cfe None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.904876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:33.932793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73683918-74ae-4397-83b8-def0d2a3b378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.933388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73683918-74ae-4397-83b8-def0d2a3b378 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.933662 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73683918-74ae-4397-83b8-def0d2a3b378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.933928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73683918-74ae-4397-83b8-def0d2a3b378 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.934537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73683918-74ae-4397-83b8-def0d2a3b378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.936447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73683918-74ae-4397-83b8-def0d2a3b378 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.956736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1360/2682] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:33 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:33.960791 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b82261-8405-416a-a26a-13b448d3f8c6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:33.971824 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b82261-8405-416a-a26a-13b448d3f8c6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:33.976568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b97a0c-fad1-4392-b52a-47a3cd62c8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:33.977189 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b97a0c-fad1-4392-b52a-47a3cd62c8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:33.977465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b97a0c-fad1-4392-b52a-47a3cd62c8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:33.977735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b97a0c-fad1-4392-b52a-47a3cd62c8b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:33.978376 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b97a0c-fad1-4392-b52a-47a3cd62c8b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:33.979966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b97a0c-fad1-4392-b52a-47a3cd62c8b5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:33.990968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1323/2683] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:33.995053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73683918-74ae-4397-83b8-def0d2a3b378 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:34.004688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73683918-74ae-4397-83b8-def0d2a3b378 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:34.009233 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea4188-2c58-4e0c-94a8-ea34a9f9e94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:34.009865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea4188-2c58-4e0c-94a8-ea34a9f9e94f admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:34.010140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea4188-2c58-4e0c-94a8-ea34a9f9e94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:34.010407 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea4188-2c58-4e0c-94a8-ea34a9f9e94f admin admin] PATH_INFO: `/v3/projects/78aef268faee44bca2cbae46317773c1/users/04a0d3fd8062454989303d6fc80c0272/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:34.025408 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea4188-2c58-4e0c-94a8-ea34a9f9e94f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78aef268faee44bca2cbae46317773c1, user_id=04a0d3fd8062454989303d6fc80c0272, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:34.027967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea4188-2c58-4e0c-94a8-ea34a9f9e94f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:34.041032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1361/2684] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:33 2022] PUT /identity/v3/projects/78aef268faee44bca2cbae46317773c1/users/04a0d3fd8062454989303d6fc80c0272/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:34.044805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b97a0c-fad1-4392-b52a-47a3cd62c8b5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:34.055037 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b97a0c-fad1-4392-b52a-47a3cd62c8b5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:34.108270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2a20a-48a3-4ead-9ddd-2d4f77149184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:34.108878 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2a20a-48a3-4ead-9ddd-2d4f77149184 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:34.109151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2a20a-48a3-4ead-9ddd-2d4f77149184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:34.109963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2a20a-48a3-4ead-9ddd-2d4f77149184 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:34.110589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2a20a-48a3-4ead-9ddd-2d4f77149184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:34.112200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2a20a-48a3-4ead-9ddd-2d4f77149184 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:34.121487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1324/2685] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:34.125573 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea4188-2c58-4e0c-94a8-ea34a9f9e94f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:34.135305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea4188-2c58-4e0c-94a8-ea34a9f9e94f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:34.139769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003972c-a4ff-47e3-bdcd-be18d32f13ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:34.140422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003972c-a4ff-47e3-bdcd-be18d32f13ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:34.140727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003972c-a4ff-47e3-bdcd-be18d32f13ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:34.140994 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003972c-a4ff-47e3-bdcd-be18d32f13ca admin admin] PATH_INFO: `/v3/projects/78aef268faee44bca2cbae46317773c1/users/04a0d3fd8062454989303d6fc80c0272/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:34.142844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003972c-a4ff-47e3-bdcd-be18d32f13ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=78aef268faee44bca2cbae46317773c1, user_id=04a0d3fd8062454989303d6fc80c0272, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:34.145459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003972c-a4ff-47e3-bdcd-be18d32f13ca admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:34.155488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1362/2686] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:34 2022] PUT /identity/v3/projects/78aef268faee44bca2cbae46317773c1/users/04a0d3fd8062454989303d6fc80c0272/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:34.166981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417d8f-9ab1-4aa7-9146-f603548a4170 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:34.167265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417d8f-9ab1-4aa7-9146-f603548a4170 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:34.167537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417d8f-9ab1-4aa7-9146-f603548a4170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:34.195184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-26417d8f-9ab1-4aa7-9146-f603548a4170 None None] MFA Rules not processed for user `04a0d3fd8062454989303d6fc80c0272`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:34.196514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-26417d8f-9ab1-4aa7-9146-f603548a4170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:34.204510 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:34.263908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1325/2687] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:34.275409 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003972c-a4ff-47e3-bdcd-be18d32f13ca admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:34.277398 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7003972c-a4ff-47e3-bdcd-be18d32f13ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:34.336308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003972c-a4ff-47e3-bdcd-be18d32f13ca admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:34.339912 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884100a-8464-492e-8c80-4c752e71bd1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:34.340607 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884100a-8464-492e-8c80-4c752e71bd1c service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:34.340882 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884100a-8464-492e-8c80-4c752e71bd1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:34.341149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884100a-8464-492e-8c80-4c752e71bd1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:34.396223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884100a-8464-492e-8c80-4c752e71bd1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:34.397613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884100a-8464-492e-8c80-4c752e71bd1c service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:34.406634 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1363/2688] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:35.903198 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26417d8f-9ab1-4aa7-9146-f603548a4170 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:35.904650 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-26417d8f-9ab1-4aa7-9146-f603548a4170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:35.917443 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26417d8f-9ab1-4aa7-9146-f603548a4170 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:35.921117 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99c2aa-662a-4aca-a648-e2492266b794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:35.921826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99c2aa-662a-4aca-a648-e2492266b794 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:35.922106 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99c2aa-662a-4aca-a648-e2492266b794 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:35.922377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99c2aa-662a-4aca-a648-e2492266b794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:35.986342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99c2aa-662a-4aca-a648-e2492266b794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:35.987770 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99c2aa-662a-4aca-a648-e2492266b794 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:35.996926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1326/2689] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:36.555283 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884100a-8464-492e-8c80-4c752e71bd1c service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:36.564254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884100a-8464-492e-8c80-4c752e71bd1c service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:36.567919 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3393e5-23dc-4428-b9e4-443fe1bfbb5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:36.568542 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3393e5-23dc-4428-b9e4-443fe1bfbb5b service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:36.568818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3393e5-23dc-4428-b9e4-443fe1bfbb5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:36.569081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3393e5-23dc-4428-b9e4-443fe1bfbb5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:36.578525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3393e5-23dc-4428-b9e4-443fe1bfbb5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:36.580009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3393e5-23dc-4428-b9e4-443fe1bfbb5b service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:36.589782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1364/2690] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:39.982991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8d17b-55a5-47d5-a303-566aa6f4f307 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:39.983447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8d17b-55a5-47d5-a303-566aa6f4f307 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:39.983555 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8d17b-55a5-47d5-a303-566aa6f4f307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:40.002648 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-edb8d17b-55a5-47d5-a303-566aa6f4f307 None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:40.003816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-edb8d17b-55a5-47d5-a303-566aa6f4f307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:40.057545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1327/2691] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:40:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:40.102921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3393e5-23dc-4428-b9e4-443fe1bfbb5b service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:40.104316 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0d3393e5-23dc-4428-b9e4-443fe1bfbb5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:40.134632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:40.184257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3393e5-23dc-4428-b9e4-443fe1bfbb5b service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:40.189198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb63c7-9403-498b-b601-be0afaa09655 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:40.189790 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb63c7-9403-498b-b601-be0afaa09655 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:40.190069 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb63c7-9403-498b-b601-be0afaa09655 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:40.190370 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb63c7-9403-498b-b601-be0afaa09655 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:40.199279 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb63c7-9403-498b-b601-be0afaa09655 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:40.200886 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb63c7-9403-498b-b601-be0afaa09655 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:40.209177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1365/2692] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:40:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:40:40.467691 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8d17b-55a5-47d5-a303-566aa6f4f307 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:40.519826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:40.536194 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8d17b-55a5-47d5-a303-566aa6f4f307 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:40.540981 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec6781-e41e-4314-836b-a8774ce056be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:40.541613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec6781-e41e-4314-836b-a8774ce056be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:40.541892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec6781-e41e-4314-836b-a8774ce056be admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:40.542167 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec6781-e41e-4314-836b-a8774ce056be admin admin] PATH_INFO: `/v3/users/3e1f2741769346d6aa664815474ec749` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:40.543307 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec6781-e41e-4314-836b-a8774ce056be admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e1f2741769346d6aa664815474ec749)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:40.545261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec6781-e41e-4314-836b-a8774ce056be admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:40.594514 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d3ec6781-e41e-4314-836b-a8774ce056be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e1f2741769346d6aa664815474ec749'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:40.602266 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d3ec6781-e41e-4314-836b-a8774ce056be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e1f2741769346d6aa664815474ec749'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:40.602932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d3ec6781-e41e-4314-836b-a8774ce056be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e1f2741769346d6aa664815474ec749'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:40.610259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d3ec6781-e41e-4314-836b-a8774ce056be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e1f2741769346d6aa664815474ec749'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:40.622290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d3ec6781-e41e-4314-836b-a8774ce056be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e1f2741769346d6aa664815474ec749'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:40.628923 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1328/2693] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:40 2022] DELETE /identity/v3/users/3e1f2741769346d6aa664815474ec749 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:40.633297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a96dc-83bb-4585-a97c-fb80ef0285ff None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:40.633646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a96dc-83bb-4585-a97c-fb80ef0285ff None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:40.634031 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a96dc-83bb-4585-a97c-fb80ef0285ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:40.653319 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-972a96dc-83bb-4585-a97c-fb80ef0285ff None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:40.654584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-972a96dc-83bb-4585-a97c-fb80ef0285ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:40.715692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037eaec-35b0-4648-88a9-ebfef00d823b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:40.715973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037eaec-35b0-4648-88a9-ebfef00d823b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:40.716243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037eaec-35b0-4648-88a9-ebfef00d823b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:40.720306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1366/2694] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:40.729962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a96dc-83bb-4585-a97c-fb80ef0285ff None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:40.731347 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-972a96dc-83bb-4585-a97c-fb80ef0285ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:40.735912 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6037eaec-35b0-4648-88a9-ebfef00d823b None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:40.737220 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6037eaec-35b0-4648-88a9-ebfef00d823b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:40.789337 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a96dc-83bb-4585-a97c-fb80ef0285ff None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:40.792885 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40396718-5412-454f-8496-07de7addae73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:40.793488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40396718-5412-454f-8496-07de7addae73 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:40.793760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40396718-5412-454f-8496-07de7addae73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:40.794028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40396718-5412-454f-8496-07de7addae73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:40.798562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1329/2695] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 30 19:40:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:40.803077 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40396718-5412-454f-8496-07de7addae73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:40.804526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40396718-5412-454f-8496-07de7addae73 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:40.812380 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037eaec-35b0-4648-88a9-ebfef00d823b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:40.814292 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6037eaec-35b0-4648-88a9-ebfef00d823b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:40.814763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1367/2696] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:40.885774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037eaec-35b0-4648-88a9-ebfef00d823b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:40.890773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a45b70-ad70-4d64-8e55-3f55d66d313c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:40.891377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a45b70-ad70-4d64-8e55-3f55d66d313c service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:40.892268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a45b70-ad70-4d64-8e55-3f55d66d313c service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:40.892571 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a45b70-ad70-4d64-8e55-3f55d66d313c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:40.901433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a45b70-ad70-4d64-8e55-3f55d66d313c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:40.903018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a45b70-ad70-4d64-8e55-3f55d66d313c service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:40.912416 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1330/2697] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:40:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 30 19:40:41.081526 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40396718-5412-454f-8496-07de7addae73 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:41.082957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-40396718-5412-454f-8496-07de7addae73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:41.096392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40396718-5412-454f-8496-07de7addae73 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:41.116093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:41.116746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:41.116997 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:41.117368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] PATH_INFO: `/v3/projects/b85f707c20d14e8c8f71c1eccd6c6d87` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:41.118800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b85f707c20d14e8c8f71c1eccd6c6d87)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:41.120904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:41.150654 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64002a5-feb7-4c12-a497-c34b7082aad2 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:41.151277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64002a5-feb7-4c12-a497-c34b7082aad2 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:41.151277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64002a5-feb7-4c12-a497-c34b7082aad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:41.159657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:41.171772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:41.174073 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-b64002a5-feb7-4c12-a497-c34b7082aad2 None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:41.175286 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b64002a5-feb7-4c12-a497-c34b7082aad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:41.176715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b85f707c20d14e8c8f71c1eccd6c6d87', 'request_id': 'req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.182533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b85f707c20d14e8c8f71c1eccd6c6d87', 'request_id': 'req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.189234 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b85f707c20d14e8c8f71c1eccd6c6d87', 'request_id': 'req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.202657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] The token cache is being invalidate because project b85f707c20d14e8c8f71c1eccd6c6d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:41.203531 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.204246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.205755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1368/2698] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:41 2022] DELETE /identity/v3/projects/b85f707c20d14e8c8f71c1eccd6c6d87 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:41.209654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:41.211114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:41.236506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1331/2699] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:40:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:41.249838 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64002a5-feb7-4c12-a497-c34b7082aad2 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:41.251778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-b64002a5-feb7-4c12-a497-c34b7082aad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:41.279788 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9380ce-94b9-482a-b038-2ee93f2ff0a8 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:41.284493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f6109-2037-45c2-8bf3-8f44bc660f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:41.285120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f6109-2037-45c2-8bf3-8f44bc660f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:41.285649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f6109-2037-45c2-8bf3-8f44bc660f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:41.285921 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f6109-2037-45c2-8bf3-8f44bc660f06 admin admin] PATH_INFO: `/v3/users/81fc09dae84148fcbb42760d115ac870` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:41.287050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f6109-2037-45c2-8bf3-8f44bc660f06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81fc09dae84148fcbb42760d115ac870)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:41.288992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f6109-2037-45c2-8bf3-8f44bc660f06 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:41.326253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64002a5-feb7-4c12-a497-c34b7082aad2 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:41.331397 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e00ab-e2f3-4f93-b26d-8fd48434dbd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:41.332003 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e00ab-e2f3-4f93-b26d-8fd48434dbd5 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:41.332273 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e00ab-e2f3-4f93-b26d-8fd48434dbd5 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:41.332613 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e00ab-e2f3-4f93-b26d-8fd48434dbd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:41.336204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-815f6109-2037-45c2-8bf3-8f44bc660f06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81fc09dae84148fcbb42760d115ac870'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.340901 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e00ab-e2f3-4f93-b26d-8fd48434dbd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:41.342489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e00ab-e2f3-4f93-b26d-8fd48434dbd5 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:41.343481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-815f6109-2037-45c2-8bf3-8f44bc660f06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81fc09dae84148fcbb42760d115ac870'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.344164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-815f6109-2037-45c2-8bf3-8f44bc660f06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81fc09dae84148fcbb42760d115ac870'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.351536 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1332/2700] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:40:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 30 19:40:41.353221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-815f6109-2037-45c2-8bf3-8f44bc660f06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81fc09dae84148fcbb42760d115ac870'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.367362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-815f6109-2037-45c2-8bf3-8f44bc660f06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81fc09dae84148fcbb42760d115ac870'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.373717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1369/2701] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:41 2022] DELETE /identity/v3/users/81fc09dae84148fcbb42760d115ac870 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:41.661621 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e00ab-e2f3-4f93-b26d-8fd48434dbd5 service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:41.692996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e00ab-e2f3-4f93-b26d-8fd48434dbd5 service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:41.698613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b21ec8-4f2f-4b84-a7c7-38c8fb13f73c None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:41.699154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b21ec8-4f2f-4b84-a7c7-38c8fb13f73c None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:41.699498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b21ec8-4f2f-4b84-a7c7-38c8fb13f73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:41.721488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-99b21ec8-4f2f-4b84-a7c7-38c8fb13f73c None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:41.723063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-99b21ec8-4f2f-4b84-a7c7-38c8fb13f73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:41.753056 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:41.753681 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:41.753955 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:41.754221 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] PATH_INFO: `/v3/projects/961152926f50440b80223ae7f1417669` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:41.755346 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] RBAC: Authorizing `identity:delete_project(project_id=961152926f50440b80223ae7f1417669)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:41.757263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:41.768941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:41.780605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1370/2702] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 30 19:40:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:41.793647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b21ec8-4f2f-4b84-a7c7-38c8fb13f73c None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:41.794993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-99b21ec8-4f2f-4b84-a7c7-38c8fb13f73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:41.802439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:41.810788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '961152926f50440b80223ae7f1417669', 'request_id': 'req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.816319 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '961152926f50440b80223ae7f1417669', 'request_id': 'req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.822541 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '961152926f50440b80223ae7f1417669', 'request_id': 'req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.837274 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] The token cache is being invalidate because project 961152926f50440b80223ae7f1417669 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:41.837561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.838183 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.839660 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1333/2703] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:41 2022] DELETE /identity/v3/projects/961152926f50440b80223ae7f1417669 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:41.843560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:41.845466 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:41.867024 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b21ec8-4f2f-4b84-a7c7-38c8fb13f73c None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:41.915628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83caff-188c-4ebd-a9c8-1b8ceb3ec43f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:41.922433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfad8f-2039-4c0e-95a6-b52c6dc3e16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:41.923643 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfad8f-2039-4c0e-95a6-b52c6dc3e16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:41.924110 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfad8f-2039-4c0e-95a6-b52c6dc3e16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:41.924400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfad8f-2039-4c0e-95a6-b52c6dc3e16f admin admin] PATH_INFO: `/v3/users/04a0d3fd8062454989303d6fc80c0272` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:41.925615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfad8f-2039-4c0e-95a6-b52c6dc3e16f admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a0d3fd8062454989303d6fc80c0272)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:41.927964 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfad8f-2039-4c0e-95a6-b52c6dc3e16f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:41.930951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c80f3-b612-412b-83cc-157009e35ee4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:41.931593 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c80f3-b612-412b-83cc-157009e35ee4 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:41.931881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c80f3-b612-412b-83cc-157009e35ee4 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:41.932204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c80f3-b612-412b-83cc-157009e35ee4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:41.934171 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4d9c80f3-b612-412b-83cc-157009e35ee4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:41.979566 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d2dfad8f-2039-4c0e-95a6-b52c6dc3e16f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a0d3fd8062454989303d6fc80c0272'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.986793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d2dfad8f-2039-4c0e-95a6-b52c6dc3e16f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a0d3fd8062454989303d6fc80c0272'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.987453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d2dfad8f-2039-4c0e-95a6-b52c6dc3e16f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a0d3fd8062454989303d6fc80c0272'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.994990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d2dfad8f-2039-4c0e-95a6-b52c6dc3e16f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a0d3fd8062454989303d6fc80c0272'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:41.998436 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c80f3-b612-412b-83cc-157009e35ee4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:42.000016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c80f3-b612-412b-83cc-157009e35ee4 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:42.006926 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d2dfad8f-2039-4c0e-95a6-b52c6dc3e16f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a0d3fd8062454989303d6fc80c0272'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:42.008114 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1371/2704] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:40:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 30 19:40:42.014423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1334/2705] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:41 2022] DELETE /identity/v3/users/04a0d3fd8062454989303d6fc80c0272 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:42.291766 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c80f3-b612-412b-83cc-157009e35ee4 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:42.303634 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c80f3-b612-412b-83cc-157009e35ee4 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:42.352713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=100 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:42.373767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:42.376760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:42.376760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:42.376760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] PATH_INFO: `/v3/projects/78aef268faee44bca2cbae46317773c1` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:42.377261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=78aef268faee44bca2cbae46317773c1)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:42.379193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:42.454180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78aef268faee44bca2cbae46317773c1', 'request_id': 'req-06791e29-352a-41fa-9ce5-629e7bf131ba'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:42.461740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78aef268faee44bca2cbae46317773c1', 'request_id': 'req-06791e29-352a-41fa-9ce5-629e7bf131ba'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:42.471459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78aef268faee44bca2cbae46317773c1', 'request_id': 'req-06791e29-352a-41fa-9ce5-629e7bf131ba'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:42.503045 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] The token cache is being invalidate because project 78aef268faee44bca2cbae46317773c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:42.503252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:42.504440 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-06791e29-352a-41fa-9ce5-629e7bf131ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:42.506263 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1372/2706] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:42 2022] DELETE /identity/v3/projects/78aef268faee44bca2cbae46317773c1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:42.512032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9a9b7-06fb-4cc2-9e7f-62b7f86f426b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:42.513115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9a9b7-06fb-4cc2-9e7f-62b7f86f426b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:42.513388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9a9b7-06fb-4cc2-9e7f-62b7f86f426b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:42.548764 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9ea9a9b7-06fb-4cc2-9e7f-62b7f86f426b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:42.548764 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ea9a9b7-06fb-4cc2-9e7f-62b7f86f426b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:42.585477 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:42.604004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1335/2707] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:42.612278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23336606-27e0-4d4e-a09b-fd97350eb9cf None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:42.612584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23336606-27e0-4d4e-a09b-fd97350eb9cf None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:42.612851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23336606-27e0-4d4e-a09b-fd97350eb9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:42.634696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-23336606-27e0-4d4e-a09b-fd97350eb9cf None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:42.635965 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-23336606-27e0-4d4e-a09b-fd97350eb9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:42.645755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1373/2708] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:42.650225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9a9b7-06fb-4cc2-9e7f-62b7f86f426b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:42.662140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9a9b7-06fb-4cc2-9e7f-62b7f86f426b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:42.667239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6a3c6-821b-4fa1-8d37-ff5ced2ef3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:42.667835 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a3c6-821b-4fa1-8d37-ff5ced2ef3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:42.668129 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a3c6-821b-4fa1-8d37-ff5ced2ef3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:42.668625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a3c6-821b-4fa1-8d37-ff5ced2ef3d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:42.669337 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6a3c6-821b-4fa1-8d37-ff5ced2ef3d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:42.669651 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6a3c6-821b-4fa1-8d37-ff5ced2ef3d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:42.671253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6a3c6-821b-4fa1-8d37-ff5ced2ef3d7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:42.692996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23336606-27e0-4d4e-a09b-fd97350eb9cf None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:42.699937 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1336/2709] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 19:40:42.707093 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23336606-27e0-4d4e-a09b-fd97350eb9cf None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:42.712933 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbff55-782e-4a30-a096-56f5ebd9abe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:42.713646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbff55-782e-4a30-a096-56f5ebd9abe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:42.713918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbff55-782e-4a30-a096-56f5ebd9abe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:42.714313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbff55-782e-4a30-a096-56f5ebd9abe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:42.715053 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbff55-782e-4a30-a096-56f5ebd9abe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:42.720138 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbff55-782e-4a30-a096-56f5ebd9abe0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:42.764300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1374/2710] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:42 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:42.769184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6a3c6-821b-4fa1-8d37-ff5ced2ef3d7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:42.782857 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6a3c6-821b-4fa1-8d37-ff5ced2ef3d7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:42.856281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948af30-6668-49f0-a918-1b49470db430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:42.857526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948af30-6668-49f0-a918-1b49470db430 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:42.857526 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948af30-6668-49f0-a918-1b49470db430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:42.857720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948af30-6668-49f0-a918-1b49470db430 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:42.858387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948af30-6668-49f0-a918-1b49470db430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:42.861213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948af30-6668-49f0-a918-1b49470db430 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:42.888763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1337/2711] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:42 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:42.893063 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbff55-782e-4a30-a096-56f5ebd9abe0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:42.921379 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbff55-782e-4a30-a096-56f5ebd9abe0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:42.926692 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78429dc7-f6bf-4e12-929e-cea2091facce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:42.927296 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78429dc7-f6bf-4e12-929e-cea2091facce admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:42.927577 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78429dc7-f6bf-4e12-929e-cea2091facce admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:42.927857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78429dc7-f6bf-4e12-929e-cea2091facce admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:42.928650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78429dc7-f6bf-4e12-929e-cea2091facce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:42.930288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78429dc7-f6bf-4e12-929e-cea2091facce admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:42.940820 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1375/2712] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:42.944756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948af30-6668-49f0-a918-1b49470db430 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:42.956235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948af30-6668-49f0-a918-1b49470db430 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:42.959732 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792d8a-cf1f-47cd-b7d6-3018917bc3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:42.960326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792d8a-cf1f-47cd-b7d6-3018917bc3be admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:42.960631 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792d8a-cf1f-47cd-b7d6-3018917bc3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:42.960899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792d8a-cf1f-47cd-b7d6-3018917bc3be admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:42.961522 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792d8a-cf1f-47cd-b7d6-3018917bc3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:42.963897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792d8a-cf1f-47cd-b7d6-3018917bc3be admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:42.975674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1338/2713] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:42.981390 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78429dc7-f6bf-4e12-929e-cea2091facce admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:42.991445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78429dc7-f6bf-4e12-929e-cea2091facce admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:42.996730 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490a443-b3ae-42c2-8f38-69a14a009acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:42.997383 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a443-b3ae-42c2-8f38-69a14a009acf admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:42.997657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a443-b3ae-42c2-8f38-69a14a009acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:42.997928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a443-b3ae-42c2-8f38-69a14a009acf admin admin] PATH_INFO: `/v3/projects/dc4504816d4a498a9c7f791da65ebd02/users/0d1893d7e8ec4d9a8399135025a2713b/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:43.013158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490a443-b3ae-42c2-8f38-69a14a009acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc4504816d4a498a9c7f791da65ebd02, user_id=0d1893d7e8ec4d9a8399135025a2713b, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:43.015686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490a443-b3ae-42c2-8f38-69a14a009acf admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:43.033163 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1376/2714] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:42 2022] PUT /identity/v3/projects/dc4504816d4a498a9c7f791da65ebd02/users/0d1893d7e8ec4d9a8399135025a2713b/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:43.044396 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af90980-ad04-4d03-b2b1-01e4985d4f31 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:43.044691 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af90980-ad04-4d03-b2b1-01e4985d4f31 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:43.044966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af90980-ad04-4d03-b2b1-01e4985d4f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:43.075222 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4af90980-ad04-4d03-b2b1-01e4985d4f31 None None] MFA Rules not processed for user `0d1893d7e8ec4d9a8399135025a2713b`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:43.076445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4af90980-ad04-4d03-b2b1-01e4985d4f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:43.141372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1339/2715] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:43.153571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490a443-b3ae-42c2-8f38-69a14a009acf admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:43.155617 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2490a443-b3ae-42c2-8f38-69a14a009acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:43.234270 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490a443-b3ae-42c2-8f38-69a14a009acf admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:43.237989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03787622-37a1-448b-9435-3ca91e35681b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:43.238686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787622-37a1-448b-9435-3ca91e35681b service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:43.238963 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787622-37a1-448b-9435-3ca91e35681b service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:43.239278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787622-37a1-448b-9435-3ca91e35681b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:43.247928 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03787622-37a1-448b-9435-3ca91e35681b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:43.249463 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03787622-37a1-448b-9435-3ca91e35681b service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:43.258609 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1377/2716] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:43.286721 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af90980-ad04-4d03-b2b1-01e4985d4f31 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:43.330622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:43.350296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af90980-ad04-4d03-b2b1-01e4985d4f31 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:43.354815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:43.355468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:43.355746 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:43.356017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] PATH_INFO: `/v3/users/0d1893d7e8ec4d9a8399135025a2713b` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:43.357188 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d1893d7e8ec4d9a8399135025a2713b)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:43.359115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:43.405027 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d1893d7e8ec4d9a8399135025a2713b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:43.413896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d1893d7e8ec4d9a8399135025a2713b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:43.414560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d1893d7e8ec4d9a8399135025a2713b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:43.422018 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d1893d7e8ec4d9a8399135025a2713b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:43.435270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d1893d7e8ec4d9a8399135025a2713b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:43.441317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1340/2717] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:43 2022] DELETE /identity/v3/users/0d1893d7e8ec4d9a8399135025a2713b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:43.445710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a799fe7-65f2-426f-abb1-22a226a99362 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:43.446193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a799fe7-65f2-426f-abb1-22a226a99362 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:43.446308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a799fe7-65f2-426f-abb1-22a226a99362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:43.466540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-4a799fe7-65f2-426f-abb1-22a226a99362 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:43.467737 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4a799fe7-65f2-426f-abb1-22a226a99362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:43.472962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:43.528361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1378/2718] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:43.538841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:43.540240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:43.597301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b515314-8ae7-4658-863d-a9ec0aa0f05b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:43.600884 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b0cd8-fa49-4bf6-8bfc-cee29bcf21f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:43.601479 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b0cd8-fa49-4bf6-8bfc-cee29bcf21f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:43.601752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b0cd8-fa49-4bf6-8bfc-cee29bcf21f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:43.602017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b0cd8-fa49-4bf6-8bfc-cee29bcf21f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:43.610694 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b0cd8-fa49-4bf6-8bfc-cee29bcf21f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:43.612107 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b0cd8-fa49-4bf6-8bfc-cee29bcf21f1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:43.621411 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1341/2719] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:43.883572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a799fe7-65f2-426f-abb1-22a226a99362 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:43.885049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4a799fe7-65f2-426f-abb1-22a226a99362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:43.898779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a799fe7-65f2-426f-abb1-22a226a99362 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:43.903524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:43.904151 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:43.904483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:43.904752 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] PATH_INFO: `/v3/projects/dc4504816d4a498a9c7f791da65ebd02` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:43.905888 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc4504816d4a498a9c7f791da65ebd02)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:43.907818 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:43.956349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc4504816d4a498a9c7f791da65ebd02', 'request_id': 'req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:43.962340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc4504816d4a498a9c7f791da65ebd02', 'request_id': 'req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:43.968575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc4504816d4a498a9c7f791da65ebd02', 'request_id': 'req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:43.980754 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] The token cache is being invalidate because project dc4504816d4a498a9c7f791da65ebd02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:43.981066 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:43.981677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-5fa97ff0-f8dc-4ccc-81c9-596acc436f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:43.983169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1379/2720] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:43 2022] DELETE /identity/v3/projects/dc4504816d4a498a9c7f791da65ebd02 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:43.988556 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e214c-7d58-489c-bf86-88f0c217f903 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:43.988846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e214c-7d58-489c-bf86-88f0c217f903 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:43.989122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e214c-7d58-489c-bf86-88f0c217f903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.010524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ec9e214c-7d58-489c-bf86-88f0c217f903 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:44.011731 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ec9e214c-7d58-489c-bf86-88f0c217f903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:44.071624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1342/2721] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:44.134148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35f78b-63fe-43fb-a7fd-8353022b990b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.134426 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35f78b-63fe-43fb-a7fd-8353022b990b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.134696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35f78b-63fe-43fb-a7fd-8353022b990b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.153991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-8c35f78b-63fe-43fb-a7fd-8353022b990b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:44.155192 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8c35f78b-63fe-43fb-a7fd-8353022b990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:44.163226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1380/2722] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:44.167535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e214c-7d58-489c-bf86-88f0c217f903 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:44.174025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:44.178287 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e214c-7d58-489c-bf86-88f0c217f903 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:44.184134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e8750-bba0-4129-996f-5dce69740452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:44.185002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e8750-bba0-4129-996f-5dce69740452 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.185517 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e8750-bba0-4129-996f-5dce69740452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.185945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e8750-bba0-4129-996f-5dce69740452 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.186586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e8750-bba0-4129-996f-5dce69740452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:44.186892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e8750-bba0-4129-996f-5dce69740452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:44.189168 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e8750-bba0-4129-996f-5dce69740452 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:44.201136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1343/2723] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:40:44.205433 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35f78b-63fe-43fb-a7fd-8353022b990b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:44.214492 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35f78b-63fe-43fb-a7fd-8353022b990b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:44.218976 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae0f47-5dcd-4311-8236-618d3916c809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:44.219584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae0f47-5dcd-4311-8236-618d3916c809 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.219834 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae0f47-5dcd-4311-8236-618d3916c809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.220094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae0f47-5dcd-4311-8236-618d3916c809 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.220750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae0f47-5dcd-4311-8236-618d3916c809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:44.222621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae0f47-5dcd-4311-8236-618d3916c809 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:44.228139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e8750-bba0-4129-996f-5dce69740452 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:44.229977 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7a8e8750-bba0-4129-996f-5dce69740452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:44.243865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e8750-bba0-4129-996f-5dce69740452 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:44.248675 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821cc4b-f22f-42e9-9a85-87842eb19d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:44.249297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821cc4b-f22f-42e9-9a85-87842eb19d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.249573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821cc4b-f22f-42e9-9a85-87842eb19d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.249840 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821cc4b-f22f-42e9-9a85-87842eb19d52 admin admin] PATH_INFO: `/v3/users/7ac8dd4bf0f44c768dd9ef0ef1b6f9a6` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.251112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821cc4b-f22f-42e9-9a85-87842eb19d52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ac8dd4bf0f44c768dd9ef0ef1b6f9a6)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:44.251761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1381/2724] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:44 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:40:44.254102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821cc4b-f22f-42e9-9a85-87842eb19d52 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:44.255632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae0f47-5dcd-4311-8236-618d3916c809 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:44.304915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2821cc4b-f22f-42e9-9a85-87842eb19d52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ac8dd4bf0f44c768dd9ef0ef1b6f9a6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:44.311898 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2821cc4b-f22f-42e9-9a85-87842eb19d52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ac8dd4bf0f44c768dd9ef0ef1b6f9a6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:44.312595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2821cc4b-f22f-42e9-9a85-87842eb19d52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ac8dd4bf0f44c768dd9ef0ef1b6f9a6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:44.320907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2821cc4b-f22f-42e9-9a85-87842eb19d52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ac8dd4bf0f44c768dd9ef0ef1b6f9a6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:44.323765 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae0f47-5dcd-4311-8236-618d3916c809 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:44.328586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fd7e8-cfd6-4566-8da2-678eb040dcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:44.329195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fd7e8-cfd6-4566-8da2-678eb040dcaf admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.329493 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fd7e8-cfd6-4566-8da2-678eb040dcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.329786 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fd7e8-cfd6-4566-8da2-678eb040dcaf admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.330507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fd7e8-cfd6-4566-8da2-678eb040dcaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:44.332574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fd7e8-cfd6-4566-8da2-678eb040dcaf admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:44.333780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-2821cc4b-f22f-42e9-9a85-87842eb19d52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ac8dd4bf0f44c768dd9ef0ef1b6f9a6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:44.339599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1344/2725] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:44 2022] DELETE /identity/v3/users/7ac8dd4bf0f44c768dd9ef0ef1b6f9a6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:44.343418 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:44.343972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53809e5-f977-4613-8e0c-29b938b83751 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.344258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53809e5-f977-4613-8e0c-29b938b83751 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.344561 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53809e5-f977-4613-8e0c-29b938b83751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.357692 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1382/2726] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:44 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:40:44.362140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fd7e8-cfd6-4566-8da2-678eb040dcaf admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:44.363773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-636fd7e8-cfd6-4566-8da2-678eb040dcaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:44.365946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f53809e5-f977-4613-8e0c-29b938b83751 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:44.367120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f53809e5-f977-4613-8e0c-29b938b83751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:44.376075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fd7e8-cfd6-4566-8da2-678eb040dcaf admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:44.435751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e95b06-6ac9-442a-9a04-bac2a0d20847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:44.436391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e95b06-6ac9-442a-9a04-bac2a0d20847 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.436727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e95b06-6ac9-442a-9a04-bac2a0d20847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.437006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e95b06-6ac9-442a-9a04-bac2a0d20847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.437649 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e95b06-6ac9-442a-9a04-bac2a0d20847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:44.439245 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e95b06-6ac9-442a-9a04-bac2a0d20847 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:44.440271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1345/2727] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:44.450945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53809e5-f977-4613-8e0c-29b938b83751 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:44.452356 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1383/2728] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:44.452849 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f53809e5-f977-4613-8e0c-29b938b83751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:44.456101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e95b06-6ac9-442a-9a04-bac2a0d20847 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:44.467213 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e95b06-6ac9-442a-9a04-bac2a0d20847 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:44.472659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d06e6-26e7-4ea2-99a0-d817d1183306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:44.473163 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d06e6-26e7-4ea2-99a0-d817d1183306 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.473441 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d06e6-26e7-4ea2-99a0-d817d1183306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.473713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d06e6-26e7-4ea2-99a0-d817d1183306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.474348 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d06e6-26e7-4ea2-99a0-d817d1183306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:44.475929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d06e6-26e7-4ea2-99a0-d817d1183306 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:44.487580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1384/2729] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:40:44.491468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d06e6-26e7-4ea2-99a0-d817d1183306 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:44.502086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d06e6-26e7-4ea2-99a0-d817d1183306 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:44.506758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27b9ce-c6a1-4137-add3-4a3e5f283b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:44.507410 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27b9ce-c6a1-4137-add3-4a3e5f283b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.507689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27b9ce-c6a1-4137-add3-4a3e5f283b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.507964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27b9ce-c6a1-4137-add3-4a3e5f283b20 admin admin] PATH_INFO: `/v3/projects/d547ff32f97940ee9afd1b80143fa853/users/835313f5f6724d29bb2a8900afa41fa3/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.514427 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53809e5-f977-4613-8e0c-29b938b83751 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:44.518198 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10e379-909c-4c4c-9c3f-978ffe77ffc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:44.518806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10e379-909c-4c4c-9c3f-978ffe77ffc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.519088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10e379-909c-4c4c-9c3f-978ffe77ffc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.519362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10e379-909c-4c4c-9c3f-978ffe77ffc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.523308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27b9ce-c6a1-4137-add3-4a3e5f283b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d547ff32f97940ee9afd1b80143fa853, user_id=835313f5f6724d29bb2a8900afa41fa3, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:44.526071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27b9ce-c6a1-4137-add3-4a3e5f283b20 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:44.529074 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10e379-909c-4c4c-9c3f-978ffe77ffc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:44.532026 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10e379-909c-4c4c-9c3f-978ffe77ffc8 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:44.542756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1385/2730] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:44 2022] PUT /identity/v3/projects/d547ff32f97940ee9afd1b80143fa853/users/835313f5f6724d29bb2a8900afa41fa3/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:44.545305 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1346/2731] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:40:44.553378 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd3e2-9b68-463a-886b-c4eba06c9839 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.553663 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd3e2-9b68-463a-886b-c4eba06c9839 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.553938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd3e2-9b68-463a-886b-c4eba06c9839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.583202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b77dd3e2-9b68-463a-886b-c4eba06c9839 None None] MFA Rules not processed for user `835313f5f6724d29bb2a8900afa41fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:44.584504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b77dd3e2-9b68-463a-886b-c4eba06c9839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:44.650890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1386/2732] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:44 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:44.663441 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10e379-909c-4c4c-9c3f-978ffe77ffc8 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:44.664897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3d10e379-909c-4c4c-9c3f-978ffe77ffc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:44.675945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:44.728201 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10e379-909c-4c4c-9c3f-978ffe77ffc8 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:44.776983 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37733d-5478-4246-add2-2e20a212e452 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:44.777726 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37733d-5478-4246-add2-2e20a212e452 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:44.778004 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37733d-5478-4246-add2-2e20a212e452 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:44.778276 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37733d-5478-4246-add2-2e20a212e452 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:44.787590 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37733d-5478-4246-add2-2e20a212e452 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:44.789069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37733d-5478-4246-add2-2e20a212e452 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:44.798412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1347/2733] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:44 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:44.902968 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dd3e2-9b68-463a-886b-c4eba06c9839 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:44.905157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b77dd3e2-9b68-463a-886b-c4eba06c9839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:44.978621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:45.010581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dd3e2-9b68-463a-886b-c4eba06c9839 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:45.017079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:45.017728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.018135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.018417 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] PATH_INFO: `/v3/projects/fb58f1584369484f801e9fbfd0a7f6f9` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.021202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb58f1584369484f801e9fbfd0a7f6f9)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:45.023342 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:45.077983 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb58f1584369484f801e9fbfd0a7f6f9', 'request_id': 'req-f842dfb3-3f11-40ce-8225-653a6e1c91a0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:45.085632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb58f1584369484f801e9fbfd0a7f6f9', 'request_id': 'req-f842dfb3-3f11-40ce-8225-653a6e1c91a0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:45.092201 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb58f1584369484f801e9fbfd0a7f6f9', 'request_id': 'req-f842dfb3-3f11-40ce-8225-653a6e1c91a0'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:45.109486 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] The token cache is being invalidate because project fb58f1584369484f801e9fbfd0a7f6f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:45.109772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:45.113081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f842dfb3-3f11-40ce-8225-653a6e1c91a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:45.116026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1387/2734] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:44 2022] DELETE /identity/v3/projects/fb58f1584369484f801e9fbfd0a7f6f9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:45.122331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4cad1-dcd7-405f-8391-0d44cf8d4d86 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.122740 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4cad1-dcd7-405f-8391-0d44cf8d4d86 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.123017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4cad1-dcd7-405f-8391-0d44cf8d4d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.154016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-eff4cad1-dcd7-405f-8391-0d44cf8d4d86 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:45.155674 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-eff4cad1-dcd7-405f-8391-0d44cf8d4d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:45.244653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1348/2735] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:45.254624 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a29b07-1491-4249-a1c3-787b36f585c5 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.255055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a29b07-1491-4249-a1c3-787b36f585c5 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.255177 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a29b07-1491-4249-a1c3-787b36f585c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.294937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-32a29b07-1491-4249-a1c3-787b36f585c5 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:45.296193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-32a29b07-1491-4249-a1c3-787b36f585c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:45.316456 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1388/2736] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:45.330723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4cad1-dcd7-405f-8391-0d44cf8d4d86 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:45.342227 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4cad1-dcd7-405f-8391-0d44cf8d4d86 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:45.349508 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e198a-6e71-4235-b485-18db4b18bfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:45.350121 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e198a-6e71-4235-b485-18db4b18bfa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.350929 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e198a-6e71-4235-b485-18db4b18bfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.351200 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e198a-6e71-4235-b485-18db4b18bfa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.351864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e198a-6e71-4235-b485-18db4b18bfa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:45.352169 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e198a-6e71-4235-b485-18db4b18bfa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:45.355500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e198a-6e71-4235-b485-18db4b18bfa7 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:45.368661 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1349/2737] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:40:45.372932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a29b07-1491-4249-a1c3-787b36f585c5 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:45.383091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a29b07-1491-4249-a1c3-787b36f585c5 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:45.393940 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682a8c7-da3b-4e45-bc33-89a185ed3c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:45.394572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682a8c7-da3b-4e45-bc33-89a185ed3c8f admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.394844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682a8c7-da3b-4e45-bc33-89a185ed3c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.395733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682a8c7-da3b-4e45-bc33-89a185ed3c8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.395733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682a8c7-da3b-4e45-bc33-89a185ed3c8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:45.397711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682a8c7-da3b-4e45-bc33-89a185ed3c8f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:45.428326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1389/2738] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:45 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:45.432250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e198a-6e71-4235-b485-18db4b18bfa7 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:45.444542 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e198a-6e71-4235-b485-18db4b18bfa7 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:45.486726 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682a8c7-da3b-4e45-bc33-89a185ed3c8f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:45.490167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7682a8c7-da3b-4e45-bc33-89a185ed3c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:45.531466 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=100 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:45.544299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15c226-59ad-4055-8cae-03703550725f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:45.545234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15c226-59ad-4055-8cae-03703550725f admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.545588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15c226-59ad-4055-8cae-03703550725f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.546168 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15c226-59ad-4055-8cae-03703550725f admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.546673 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15c226-59ad-4055-8cae-03703550725f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:45.549408 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15c226-59ad-4055-8cae-03703550725f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:45.582100 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1350/2739] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:45 2022] POST /identity/v3/users => generated 291 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:40:45.583554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15c226-59ad-4055-8cae-03703550725f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:45.595628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15c226-59ad-4055-8cae-03703550725f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:45.601372 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588baed-e9be-4674-bb7b-b9628979f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:45.602150 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588baed-e9be-4674-bb7b-b9628979f74b admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.602432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588baed-e9be-4674-bb7b-b9628979f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.602705 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588baed-e9be-4674-bb7b-b9628979f74b admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.603412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588baed-e9be-4674-bb7b-b9628979f74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:45.605508 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588baed-e9be-4674-bb7b-b9628979f74b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:45.616047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682a8c7-da3b-4e45-bc33-89a185ed3c8f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:45.618782 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1351/2740] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:45.622733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91904c1-dc36-4e03-8643-dd6e995984a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:45.623147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588baed-e9be-4674-bb7b-b9628979f74b admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:45.623874 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91904c1-dc36-4e03-8643-dd6e995984a0 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.624576 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91904c1-dc36-4e03-8643-dd6e995984a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.625033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91904c1-dc36-4e03-8643-dd6e995984a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.627618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a91904c1-dc36-4e03-8643-dd6e995984a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:45.641972 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91904c1-dc36-4e03-8643-dd6e995984a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:45.643935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91904c1-dc36-4e03-8643-dd6e995984a0 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:45.645788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588baed-e9be-4674-bb7b-b9628979f74b admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:45.651788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12cc29-8674-4cad-88e5-b0b1b060cd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:45.652429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12cc29-8674-4cad-88e5-b0b1b060cd1a admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.652711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12cc29-8674-4cad-88e5-b0b1b060cd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.652980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12cc29-8674-4cad-88e5-b0b1b060cd1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.653647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12cc29-8674-4cad-88e5-b0b1b060cd1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:45.655618 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12cc29-8674-4cad-88e5-b0b1b060cd1a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:45.661403 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1390/2741] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:40:45.676733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1352/2742] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:45.677710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91904c1-dc36-4e03-8643-dd6e995984a0 service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:45.692317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91904c1-dc36-4e03-8643-dd6e995984a0 service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:45.700592 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e954a5-a6f8-4b2c-9d3d-b13c86c56ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:45.701349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e954a5-a6f8-4b2c-9d3d-b13c86c56ade admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.701845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e954a5-a6f8-4b2c-9d3d-b13c86c56ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.702184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e954a5-a6f8-4b2c-9d3d-b13c86c56ade admin admin] PATH_INFO: `/v3/projects/9fb45af355274456bc3759ebb76f6b72/users/c5b46acc6b57473fa4cab2e37b761748/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.725768 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e954a5-a6f8-4b2c-9d3d-b13c86c56ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb45af355274456bc3759ebb76f6b72, user_id=c5b46acc6b57473fa4cab2e37b761748, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:45.728759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e954a5-a6f8-4b2c-9d3d-b13c86c56ade admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:45.744600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1391/2743] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:45 2022] PUT /identity/v3/projects/9fb45af355274456bc3759ebb76f6b72/users/c5b46acc6b57473fa4cab2e37b761748/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:40:45.749886 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77bb5-0749-4da4-8e8a-ec594ba0299e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.750179 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77bb5-0749-4da4-8e8a-ec594ba0299e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.750464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea77bb5-0749-4da4-8e8a-ec594ba0299e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.775293 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9ea77bb5-0749-4da4-8e8a-ec594ba0299e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:45.776551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9ea77bb5-0749-4da4-8e8a-ec594ba0299e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:45.835545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1353/2744] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:45.845709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e954a5-a6f8-4b2c-9d3d-b13c86c56ade admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:45.847478 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-06e954a5-a6f8-4b2c-9d3d-b13c86c56ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:45.857202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:45.906991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e954a5-a6f8-4b2c-9d3d-b13c86c56ade admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:45.911710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be405fa-a44b-4de7-84c0-fc46630834c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:45.912335 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be405fa-a44b-4de7-84c0-fc46630834c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:45.912657 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be405fa-a44b-4de7-84c0-fc46630834c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:45.912966 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be405fa-a44b-4de7-84c0-fc46630834c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:45.923474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be405fa-a44b-4de7-84c0-fc46630834c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:45.924941 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be405fa-a44b-4de7-84c0-fc46630834c3 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:45.934738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1392/2745] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:46.772879 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbec70-29bf-4339-a18e-c172b1d9326e None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:46.773899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbec70-29bf-4339-a18e-c172b1d9326e None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:46.774454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbec70-29bf-4339-a18e-c172b1d9326e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:46.809968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a7fbec70-29bf-4339-a18e-c172b1d9326e None None] MFA Rules not processed for user `c5b46acc6b57473fa4cab2e37b761748`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:46.811300 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a7fbec70-29bf-4339-a18e-c172b1d9326e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:46.913993 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1354/2746] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:46.927904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be405fa-a44b-4de7-84c0-fc46630834c3 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:46.931742 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0be405fa-a44b-4de7-84c0-fc46630834c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:47.003575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be405fa-a44b-4de7-84c0-fc46630834c3 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:47.009084 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f53511-445d-4c7c-a862-f24ffab288db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:47.009857 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f53511-445d-4c7c-a862-f24ffab288db service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:47.010038 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f53511-445d-4c7c-a862-f24ffab288db service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:47.010219 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f53511-445d-4c7c-a862-f24ffab288db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:47.021575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f53511-445d-4c7c-a862-f24ffab288db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:47.025156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f53511-445d-4c7c-a862-f24ffab288db service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:47.035238 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1393/2747] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:51.569344 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902b275-8436-40a6-92fe-0f0e926333bc None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:51.569806 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902b275-8436-40a6-92fe-0f0e926333bc None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:51.570066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902b275-8436-40a6-92fe-0f0e926333bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:51.590742 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1902b275-8436-40a6-92fe-0f0e926333bc None None] MFA Rules not processed for user `c5b46acc6b57473fa4cab2e37b761748`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:51.592778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1902b275-8436-40a6-92fe-0f0e926333bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:51.602412 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1355/2748] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:51.834384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f53511-445d-4c7c-a862-f24ffab288db service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:51.843558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f53511-445d-4c7c-a862-f24ffab288db service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:51.847720 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54283ec-4d63-4a15-a929-b0ffdb5ac4d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:51.848312 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54283ec-4d63-4a15-a929-b0ffdb5ac4d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:51.848629 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54283ec-4d63-4a15-a929-b0ffdb5ac4d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:51.848901 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54283ec-4d63-4a15-a929-b0ffdb5ac4d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:51.850622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e54283ec-4d63-4a15-a929-b0ffdb5ac4d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:51.860252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54283ec-4d63-4a15-a929-b0ffdb5ac4d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:51.861700 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54283ec-4d63-4a15-a929-b0ffdb5ac4d7 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:51.869880 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1394/2749] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:53.777537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902b275-8436-40a6-92fe-0f0e926333bc None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:53.779095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1902b275-8436-40a6-92fe-0f0e926333bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:53.791986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902b275-8436-40a6-92fe-0f0e926333bc None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:53.795698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12e5fe-9b93-45e5-a3b9-8f5a0e1268c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:53.796312 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12e5fe-9b93-45e5-a3b9-8f5a0e1268c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:53.796622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12e5fe-9b93-45e5-a3b9-8f5a0e1268c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:53.796897 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12e5fe-9b93-45e5-a3b9-8f5a0e1268c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:53.798597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-2b12e5fe-9b93-45e5-a3b9-8f5a0e1268c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:53.810320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12e5fe-9b93-45e5-a3b9-8f5a0e1268c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:53.811688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12e5fe-9b93-45e5-a3b9-8f5a0e1268c1 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:53.821504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1356/2750] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:54.118488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54283ec-4d63-4a15-a929-b0ffdb5ac4d7 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:54.120981 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-e54283ec-4d63-4a15-a929-b0ffdb5ac4d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:54.147726 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54283ec-4d63-4a15-a929-b0ffdb5ac4d7 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:54.152305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:54.152954 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:54.153229 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:54.153497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] PATH_INFO: `/v3/users/835313f5f6724d29bb2a8900afa41fa3` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:54.154614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=835313f5f6724d29bb2a8900afa41fa3)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:54.156838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:54.214067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '835313f5f6724d29bb2a8900afa41fa3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:54.224654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '835313f5f6724d29bb2a8900afa41fa3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:54.229621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '835313f5f6724d29bb2a8900afa41fa3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:54.243065 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '835313f5f6724d29bb2a8900afa41fa3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:54.258071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '835313f5f6724d29bb2a8900afa41fa3'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:54.264050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1395/2751] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:54 2022] DELETE /identity/v3/users/835313f5f6724d29bb2a8900afa41fa3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:54.269124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88222c-2fc3-4807-96db-94ccc13d3aa8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:54.269662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88222c-2fc3-4807-96db-94ccc13d3aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:54.270085 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88222c-2fc3-4807-96db-94ccc13d3aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:54.289460 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-9a88222c-2fc3-4807-96db-94ccc13d3aa8 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:54.290896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9a88222c-2fc3-4807-96db-94ccc13d3aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:54.321930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:54.359277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1357/2752] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:40:54.373273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:54.374964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:54.390611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:54.448439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e71d6-e6ef-4ece-a6a2-841507aebd17 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:54.452068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fc8f8-a290-40ec-91b6-7eec037373f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:54.453019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fc8f8-a290-40ec-91b6-7eec037373f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:54.453019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fc8f8-a290-40ec-91b6-7eec037373f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:54.453275 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fc8f8-a290-40ec-91b6-7eec037373f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:54.463878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fc8f8-a290-40ec-91b6-7eec037373f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:54.465422 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fc8f8-a290-40ec-91b6-7eec037373f2 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:54.476489 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1396/2753] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:40:55.099685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88222c-2fc3-4807-96db-94ccc13d3aa8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:55.101738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-9a88222c-2fc3-4807-96db-94ccc13d3aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:55.113752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88222c-2fc3-4807-96db-94ccc13d3aa8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:55.119171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:55.119891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:55.120235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:55.120616 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] PATH_INFO: `/v3/projects/d547ff32f97940ee9afd1b80143fa853` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:55.121944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d547ff32f97940ee9afd1b80143fa853)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:55.123967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:55.184819 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd547ff32f97940ee9afd1b80143fa853', 'request_id': 'req-3edac52a-f27b-46bd-aed9-0dfb422a2a24'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:55.191175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd547ff32f97940ee9afd1b80143fa853', 'request_id': 'req-3edac52a-f27b-46bd-aed9-0dfb422a2a24'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:55.200197 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd547ff32f97940ee9afd1b80143fa853', 'request_id': 'req-3edac52a-f27b-46bd-aed9-0dfb422a2a24'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:55.215828 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] The token cache is being invalidate because project d547ff32f97940ee9afd1b80143fa853 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:55.216127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:55.216733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:55.227910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1358/2754] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:55 2022] DELETE /identity/v3/projects/d547ff32f97940ee9afd1b80143fa853 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:57.283141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fc8f8-a290-40ec-91b6-7eec037373f2 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:57.284915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-776fc8f8-a290-40ec-91b6-7eec037373f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:57.309132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fc8f8-a290-40ec-91b6-7eec037373f2 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:57.382121 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4c5b7-66a7-4b4b-8d3b-e19a0702afae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:57.382755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4c5b7-66a7-4b4b-8d3b-e19a0702afae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:57.383034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4c5b7-66a7-4b4b-8d3b-e19a0702afae admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:57.383331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4c5b7-66a7-4b4b-8d3b-e19a0702afae admin admin] PATH_INFO: `/v3/users/f7aa40f3966d4c97ab6a1a764722969d` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:57.384519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4c5b7-66a7-4b4b-8d3b-e19a0702afae admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7aa40f3966d4c97ab6a1a764722969d)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:57.386533 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4c5b7-66a7-4b4b-8d3b-e19a0702afae admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:57.433408 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1cb4c5b7-66a7-4b4b-8d3b-e19a0702afae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7aa40f3966d4c97ab6a1a764722969d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:57.440600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1cb4c5b7-66a7-4b4b-8d3b-e19a0702afae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7aa40f3966d4c97ab6a1a764722969d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:57.441243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1cb4c5b7-66a7-4b4b-8d3b-e19a0702afae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7aa40f3966d4c97ab6a1a764722969d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:57.448004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1cb4c5b7-66a7-4b4b-8d3b-e19a0702afae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7aa40f3966d4c97ab6a1a764722969d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:57.460265 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1cb4c5b7-66a7-4b4b-8d3b-e19a0702afae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7aa40f3966d4c97ab6a1a764722969d'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:57.466102 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1397/2755] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:40:57 2022] DELETE /identity/v3/users/f7aa40f3966d4c97ab6a1a764722969d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:57.639933 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:57.642914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:57.658435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edac52a-f27b-46bd-aed9-0dfb422a2a24 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:57.718269 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:57.719192 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:57.719676 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:57.720122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] PATH_INFO: `/v3/projects/395bb9b09f7948bbad5817c88aa4accc` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:57.721290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=395bb9b09f7948bbad5817c88aa4accc)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:57.723198 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:57.770270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '395bb9b09f7948bbad5817c88aa4accc', 'request_id': 'req-02a422ee-3e61-4cab-91a7-be0283a14b40'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:57.774780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:57.776212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '395bb9b09f7948bbad5817c88aa4accc', 'request_id': 'req-02a422ee-3e61-4cab-91a7-be0283a14b40'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:57.784113 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '395bb9b09f7948bbad5817c88aa4accc', 'request_id': 'req-02a422ee-3e61-4cab-91a7-be0283a14b40'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:57.798554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] The token cache is being invalidate because project 395bb9b09f7948bbad5817c88aa4accc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:40:57.798960 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:57.799574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-02a422ee-3e61-4cab-91a7-be0283a14b40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:40:57.801071 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1359/2756] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:40:57 2022] DELETE /identity/v3/projects/395bb9b09f7948bbad5817c88aa4accc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:57.806154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2bee4-cef5-43c2-871c-169d20ec658e None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:57.806439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2bee4-cef5-43c2-871c-169d20ec658e None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:57.806711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2bee4-cef5-43c2-871c-169d20ec658e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:57.828361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a6a2bee4-cef5-43c2-871c-169d20ec658e None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:57.829630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a6a2bee4-cef5-43c2-871c-169d20ec658e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:57.859650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:40:57.893297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1398/2757] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:57.901026 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19520c-f31e-4004-993e-fb0edfe0d5bf None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:57.901310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19520c-f31e-4004-993e-fb0edfe0d5bf None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:57.901582 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19520c-f31e-4004-993e-fb0edfe0d5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:57.921384 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4d19520c-f31e-4004-993e-fb0edfe0d5bf None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:57.922792 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4d19520c-f31e-4004-993e-fb0edfe0d5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:57.931306 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1360/2758] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:57.935611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2bee4-cef5-43c2-871c-169d20ec658e None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:57.946150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2bee4-cef5-43c2-871c-169d20ec658e None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:57.950995 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5fc2a-9161-4b54-a04c-e5c924c88734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:57.951677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5fc2a-9161-4b54-a04c-e5c924c88734 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:57.951869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5fc2a-9161-4b54-a04c-e5c924c88734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:57.952140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5fc2a-9161-4b54-a04c-e5c924c88734 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:57.952821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5fc2a-9161-4b54-a04c-e5c924c88734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:40:57.953122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5fc2a-9161-4b54-a04c-e5c924c88734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:57.955597 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5fc2a-9161-4b54-a04c-e5c924c88734 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:57.966779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1399/2759] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:40:57.971587 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19520c-f31e-4004-993e-fb0edfe0d5bf None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:57.980730 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19520c-f31e-4004-993e-fb0edfe0d5bf None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:57.985943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f51bdd-4a4b-4c68-b371-605934e0805c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:57.986533 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f51bdd-4a4b-4c68-b371-605934e0805c admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:57.986808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f51bdd-4a4b-4c68-b371-605934e0805c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:57.987077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f51bdd-4a4b-4c68-b371-605934e0805c admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:57.987700 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f51bdd-4a4b-4c68-b371-605934e0805c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:57.989634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f51bdd-4a4b-4c68-b371-605934e0805c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:58.016096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1361/2760] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:40:57 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:40:58.020333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5fc2a-9161-4b54-a04c-e5c924c88734 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:58.030313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5fc2a-9161-4b54-a04c-e5c924c88734 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:58.086025 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77effab6-cea9-4daf-b734-f64d43f6962a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:58.086671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77effab6-cea9-4daf-b734-f64d43f6962a admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:58.086952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77effab6-cea9-4daf-b734-f64d43f6962a admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:58.087226 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77effab6-cea9-4daf-b734-f64d43f6962a admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:58.087864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77effab6-cea9-4daf-b734-f64d43f6962a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:58.089807 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77effab6-cea9-4daf-b734-f64d43f6962a admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:58.111113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1400/2761] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:40:58 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:40:58.115205 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f51bdd-4a4b-4c68-b371-605934e0805c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:58.125171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f51bdd-4a4b-4c68-b371-605934e0805c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:58.129698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a14cb8-1199-44f7-b126-5ce0fdfec1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:58.130277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a14cb8-1199-44f7-b126-5ce0fdfec1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:58.130549 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a14cb8-1199-44f7-b126-5ce0fdfec1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:58.130815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a14cb8-1199-44f7-b126-5ce0fdfec1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:58.131442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a14cb8-1199-44f7-b126-5ce0fdfec1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:58.133059 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a14cb8-1199-44f7-b126-5ce0fdfec1d0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:58.142595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1362/2762] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:58.146365 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77effab6-cea9-4daf-b734-f64d43f6962a admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:58.156048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77effab6-cea9-4daf-b734-f64d43f6962a admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:58.161382 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9da0f-e32d-498b-9bb5-815f82027518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:58.161960 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9da0f-e32d-498b-9bb5-815f82027518 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:58.162235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9da0f-e32d-498b-9bb5-815f82027518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:58.162503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9da0f-e32d-498b-9bb5-815f82027518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:58.163157 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9da0f-e32d-498b-9bb5-815f82027518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:58.164738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9da0f-e32d-498b-9bb5-815f82027518 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:58.177699 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1401/2763] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:40:58.182145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a14cb8-1199-44f7-b126-5ce0fdfec1d0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:58.192177 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a14cb8-1199-44f7-b126-5ce0fdfec1d0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:58.198845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006e200-3b7f-413b-be82-7e0229203715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:58.200019 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006e200-3b7f-413b-be82-7e0229203715 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:58.200715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006e200-3b7f-413b-be82-7e0229203715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:58.201100 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006e200-3b7f-413b-be82-7e0229203715 admin admin] PATH_INFO: `/v3/projects/5b6ad3c6bda647e4af491892cebf82a1/users/d9ba795f1d4841e883e762aec293f32a/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:58.218715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006e200-3b7f-413b-be82-7e0229203715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6ad3c6bda647e4af491892cebf82a1, user_id=d9ba795f1d4841e883e762aec293f32a, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:58.221949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006e200-3b7f-413b-be82-7e0229203715 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:58.237259 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1363/2764] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:40:58 2022] PUT /identity/v3/projects/5b6ad3c6bda647e4af491892cebf82a1/users/d9ba795f1d4841e883e762aec293f32a/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:40:58.256259 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837b4bf-5fdc-497b-9f0c-7cf965578030 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:58.256650 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837b4bf-5fdc-497b-9f0c-7cf965578030 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:58.256932 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837b4bf-5fdc-497b-9f0c-7cf965578030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:58.286841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f837b4bf-5fdc-497b-9f0c-7cf965578030 None None] MFA Rules not processed for user `d9ba795f1d4841e883e762aec293f32a`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:40:58.288051 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f837b4bf-5fdc-497b-9f0c-7cf965578030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:58.352000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1402/2765] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:40:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:40:58.362947 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006e200-3b7f-413b-be82-7e0229203715 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:40:58.365373 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5006e200-3b7f-413b-be82-7e0229203715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:40:58.447240 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006e200-3b7f-413b-be82-7e0229203715 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:40:58.452774 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fee1dc-3fed-418d-b778-82a766c96b48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:40:58.453376 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fee1dc-3fed-418d-b778-82a766c96b48 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:40:58.454136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fee1dc-3fed-418d-b778-82a766c96b48 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:40:58.454407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fee1dc-3fed-418d-b778-82a766c96b48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:40:58.462073 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fee1dc-3fed-418d-b778-82a766c96b48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:40:58.463641 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fee1dc-3fed-418d-b778-82a766c96b48 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:40:58.473568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1364/2766] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:40:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:41:01.147221 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837b4bf-5fdc-497b-9f0c-7cf965578030 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:01.175876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:01.207619 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837b4bf-5fdc-497b-9f0c-7cf965578030 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:01.212047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:01.212698 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:01.212974 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:01.213240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] PATH_INFO: `/v3/users/d9ba795f1d4841e883e762aec293f32a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:01.214329 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9ba795f1d4841e883e762aec293f32a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:01.216204 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:01.259513 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9ba795f1d4841e883e762aec293f32a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:01.267236 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9ba795f1d4841e883e762aec293f32a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:01.267956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9ba795f1d4841e883e762aec293f32a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:01.274755 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9ba795f1d4841e883e762aec293f32a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:01.287666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9ba795f1d4841e883e762aec293f32a'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:01.293443 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1403/2767] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:41:01 2022] DELETE /identity/v3/users/d9ba795f1d4841e883e762aec293f32a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:01.297913 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e00dfb-29ce-407c-8f7f-aed2f65751ad None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:01.298373 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e00dfb-29ce-407c-8f7f-aed2f65751ad None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:01.298496 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e00dfb-29ce-407c-8f7f-aed2f65751ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:01.316604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-04e00dfb-29ce-407c-8f7f-aed2f65751ad None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:01.317769 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-04e00dfb-29ce-407c-8f7f-aed2f65751ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:01.375058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1365/2768] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:01.384830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:01.386269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:01.442756 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1a929-093a-4370-946e-501702c3e4d7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:01.446376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad28e6e-126e-477f-9e79-a3af262509c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:01.446970 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad28e6e-126e-477f-9e79-a3af262509c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:01.447246 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad28e6e-126e-477f-9e79-a3af262509c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:01.447516 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad28e6e-126e-477f-9e79-a3af262509c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:01.456734 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad28e6e-126e-477f-9e79-a3af262509c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:01.458147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad28e6e-126e-477f-9e79-a3af262509c9 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:01.467268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1404/2769] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:41:01.970921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e00dfb-29ce-407c-8f7f-aed2f65751ad None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:01.973054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-04e00dfb-29ce-407c-8f7f-aed2f65751ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:01.987860 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e00dfb-29ce-407c-8f7f-aed2f65751ad None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:01.993001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:01.994108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:01.994568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:01.994842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] PATH_INFO: `/v3/projects/5b6ad3c6bda647e4af491892cebf82a1` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:01.996087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b6ad3c6bda647e4af491892cebf82a1)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:01.998063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:01.999629 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:02.045930 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b6ad3c6bda647e4af491892cebf82a1', 'request_id': 'req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:02.051833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b6ad3c6bda647e4af491892cebf82a1', 'request_id': 'req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:02.057951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b6ad3c6bda647e4af491892cebf82a1', 'request_id': 'req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:02.070547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] The token cache is being invalidate because project 5b6ad3c6bda647e4af491892cebf82a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:41:02.070832 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:02.071464 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-caa8f885-d2a2-4d04-80a5-6f836ea8d78c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:02.072986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1366/2770] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:41:01 2022] DELETE /identity/v3/projects/5b6ad3c6bda647e4af491892cebf82a1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:02.077947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d1fce-ec2d-464b-8c38-5c722c94bb4b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.078224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d1fce-ec2d-464b-8c38-5c722c94bb4b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.078496 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d1fce-ec2d-464b-8c38-5c722c94bb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.098113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ee5d1fce-ec2d-464b-8c38-5c722c94bb4b None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:02.099321 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ee5d1fce-ec2d-464b-8c38-5c722c94bb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:02.149223 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:02.164939 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1405/2771] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:41:02.173290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a882560-e709-41c4-a5e3-b25206e3dd33 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.173569 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a882560-e709-41c4-a5e3-b25206e3dd33 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.173835 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a882560-e709-41c4-a5e3-b25206e3dd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.194895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-1a882560-e709-41c4-a5e3-b25206e3dd33 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:02.196512 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-1a882560-e709-41c4-a5e3-b25206e3dd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:02.205011 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1367/2772] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:02.209217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d1fce-ec2d-464b-8c38-5c722c94bb4b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.218843 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d1fce-ec2d-464b-8c38-5c722c94bb4b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.223355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b4c4b-e5c6-4234-8495-04d4e7aa4df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.223943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b4c4b-e5c6-4234-8495-04d4e7aa4df7 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.224222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b4c4b-e5c6-4234-8495-04d4e7aa4df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.224537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b4c4b-e5c6-4234-8495-04d4e7aa4df7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.225180 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b4c4b-e5c6-4234-8495-04d4e7aa4df7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:41:02.225507 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b4c4b-e5c6-4234-8495-04d4e7aa4df7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:02.227080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b4c4b-e5c6-4234-8495-04d4e7aa4df7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:02.238844 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1406/2773] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:41:02.243191 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a882560-e709-41c4-a5e3-b25206e3dd33 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.252890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a882560-e709-41c4-a5e3-b25206e3dd33 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.257378 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab00966-d3eb-4cac-92bb-5f4e3d15b63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.257958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00966-d3eb-4cac-92bb-5f4e3d15b63e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.258230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00966-d3eb-4cac-92bb-5f4e3d15b63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.258495 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00966-d3eb-4cac-92bb-5f4e3d15b63e admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.259109 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab00966-d3eb-4cac-92bb-5f4e3d15b63e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:02.261029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab00966-d3eb-4cac-92bb-5f4e3d15b63e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:02.287375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1368/2774] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:41:02 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:41:02.291878 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b4c4b-e5c6-4234-8495-04d4e7aa4df7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.302189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b4c4b-e5c6-4234-8495-04d4e7aa4df7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.356937 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b52f39-81ea-4f2d-8bd1-26046fd508f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.357537 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b52f39-81ea-4f2d-8bd1-26046fd508f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.357831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b52f39-81ea-4f2d-8bd1-26046fd508f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.358083 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b52f39-81ea-4f2d-8bd1-26046fd508f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.358705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b52f39-81ea-4f2d-8bd1-26046fd508f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:02.361227 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b52f39-81ea-4f2d-8bd1-26046fd508f2 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:02.382546 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1407/2775] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:41:02 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:41:02.386712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab00966-d3eb-4cac-92bb-5f4e3d15b63e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.397132 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab00966-d3eb-4cac-92bb-5f4e3d15b63e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.401810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34093f4a-ac91-4533-96cf-8b9d33c9d2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.402407 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34093f4a-ac91-4533-96cf-8b9d33c9d2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.402685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34093f4a-ac91-4533-96cf-8b9d33c9d2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.402956 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34093f4a-ac91-4533-96cf-8b9d33c9d2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.403589 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34093f4a-ac91-4533-96cf-8b9d33c9d2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:02.405215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34093f4a-ac91-4533-96cf-8b9d33c9d2d6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:02.417038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1369/2776] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:41:02.421034 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b52f39-81ea-4f2d-8bd1-26046fd508f2 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.431574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b52f39-81ea-4f2d-8bd1-26046fd508f2 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.436147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8b959-66fa-4b0f-9a46-f76dee8f5523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.436839 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8b959-66fa-4b0f-9a46-f76dee8f5523 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.437120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8b959-66fa-4b0f-9a46-f76dee8f5523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.437393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8b959-66fa-4b0f-9a46-f76dee8f5523 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.438028 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8b959-66fa-4b0f-9a46-f76dee8f5523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:02.439600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8b959-66fa-4b0f-9a46-f76dee8f5523 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:02.450097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1408/2777] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:41:02.453946 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34093f4a-ac91-4533-96cf-8b9d33c9d2d6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.463706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34093f4a-ac91-4533-96cf-8b9d33c9d2d6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.470108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0aa09d-31c3-4119-a4d1-c85451ebc84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.470765 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0aa09d-31c3-4119-a4d1-c85451ebc84d admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.471126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0aa09d-31c3-4119-a4d1-c85451ebc84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.471451 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0aa09d-31c3-4119-a4d1-c85451ebc84d admin admin] PATH_INFO: `/v3/projects/543f256d05a440bfa15dac02b5c0a40e/users/722a5b5ba7b144fdaa47eec75d6d9fc5/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.487380 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0aa09d-31c3-4119-a4d1-c85451ebc84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=543f256d05a440bfa15dac02b5c0a40e, user_id=722a5b5ba7b144fdaa47eec75d6d9fc5, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:02.490061 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0aa09d-31c3-4119-a4d1-c85451ebc84d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:02.504717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1370/2778] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:41:02 2022] PUT /identity/v3/projects/543f256d05a440bfa15dac02b5c0a40e/users/722a5b5ba7b144fdaa47eec75d6d9fc5/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:41:02.515385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df1a1a-9205-49e3-8b41-149efbef0702 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.515815 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df1a1a-9205-49e3-8b41-149efbef0702 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.516162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df1a1a-9205-49e3-8b41-149efbef0702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.544485 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-67df1a1a-9205-49e3-8b41-149efbef0702 None None] MFA Rules not processed for user `722a5b5ba7b144fdaa47eec75d6d9fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:02.545942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-67df1a1a-9205-49e3-8b41-149efbef0702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:02.610762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1409/2779] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:02.615263 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0aa09d-31c3-4119-a4d1-c85451ebc84d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.676727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0aa09d-31c3-4119-a4d1-c85451ebc84d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.681439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdf52c-5617-46d1-93e0-a695bb3960b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.682171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdf52c-5617-46d1-93e0-a695bb3960b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.682447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdf52c-5617-46d1-93e0-a695bb3960b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.682712 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdf52c-5617-46d1-93e0-a695bb3960b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.683371 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdf52c-5617-46d1-93e0-a695bb3960b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:02.685311 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdf52c-5617-46d1-93e0-a695bb3960b3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:02.710874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1371/2780] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:41:02 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:41:02.714865 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df1a1a-9205-49e3-8b41-149efbef0702 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.726136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df1a1a-9205-49e3-8b41-149efbef0702 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.769119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:02.782979 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19712a67-f67d-400b-9bbd-f65e92c3fa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.783581 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19712a67-f67d-400b-9bbd-f65e92c3fa10 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.783855 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19712a67-f67d-400b-9bbd-f65e92c3fa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.784119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19712a67-f67d-400b-9bbd-f65e92c3fa10 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.784772 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19712a67-f67d-400b-9bbd-f65e92c3fa10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:02.786923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19712a67-f67d-400b-9bbd-f65e92c3fa10 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:02.808104 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1410/2781] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:41:02 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:41:02.812234 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdf52c-5617-46d1-93e0-a695bb3960b3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.822571 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdf52c-5617-46d1-93e0-a695bb3960b3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.827256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5f33c-6235-4a77-b163-cef5430bc957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.827841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5f33c-6235-4a77-b163-cef5430bc957 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.828120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5f33c-6235-4a77-b163-cef5430bc957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.828417 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5f33c-6235-4a77-b163-cef5430bc957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.829576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5f33c-6235-4a77-b163-cef5430bc957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:02.831166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5f33c-6235-4a77-b163-cef5430bc957 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:02.836990 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:02.842133 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1372/2782] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:41:02.847037 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19712a67-f67d-400b-9bbd-f65e92c3fa10 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.856735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19712a67-f67d-400b-9bbd-f65e92c3fa10 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.861202 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fa499-ce6e-4295-9dbe-14b92a174aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.861831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fa499-ce6e-4295-9dbe-14b92a174aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.862119 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fa499-ce6e-4295-9dbe-14b92a174aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.862367 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fa499-ce6e-4295-9dbe-14b92a174aeb admin admin] PATH_INFO: `/v3/projects/2ef4a8df3c874dd9b53b4185bbf1dd1a/users/3fc92381d6f546abac4e8e84708df4a1/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.877477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fa499-ce6e-4295-9dbe-14b92a174aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef4a8df3c874dd9b53b4185bbf1dd1a, user_id=3fc92381d6f546abac4e8e84708df4a1, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:02.880088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fa499-ce6e-4295-9dbe-14b92a174aeb admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:02.893571 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1411/2783] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:41:02 2022] PUT /identity/v3/projects/2ef4a8df3c874dd9b53b4185bbf1dd1a/users/3fc92381d6f546abac4e8e84708df4a1/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:02.897482 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5f33c-6235-4a77-b163-cef5430bc957 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.908421 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5f33c-6235-4a77-b163-cef5430bc957 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.966050 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6a70e-075f-452b-9a55-4f3a853db635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.966688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6a70e-075f-452b-9a55-4f3a853db635 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:02.966962 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6a70e-075f-452b-9a55-4f3a853db635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:02.967229 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6a70e-075f-452b-9a55-4f3a853db635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:02.967864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6a70e-075f-452b-9a55-4f3a853db635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:02.970127 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6a70e-075f-452b-9a55-4f3a853db635 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:02.980136 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1373/2784] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:41:02.984252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fa499-ce6e-4295-9dbe-14b92a174aeb admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:02.994191 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fa499-ce6e-4295-9dbe-14b92a174aeb admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:02.999096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1399a-52fc-45ef-87d3-33c53fb55dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:02.999740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1399a-52fc-45ef-87d3-33c53fb55dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:03.000013 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1399a-52fc-45ef-87d3-33c53fb55dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:03.000278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1399a-52fc-45ef-87d3-33c53fb55dcf admin admin] PATH_INFO: `/v3/projects/2ef4a8df3c874dd9b53b4185bbf1dd1a/users/3fc92381d6f546abac4e8e84708df4a1/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:03.002214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a1399a-52fc-45ef-87d3-33c53fb55dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef4a8df3c874dd9b53b4185bbf1dd1a, user_id=3fc92381d6f546abac4e8e84708df4a1, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:03.004869 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a1399a-52fc-45ef-87d3-33c53fb55dcf admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:03.015447 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1412/2785] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:41:02 2022] PUT /identity/v3/projects/2ef4a8df3c874dd9b53b4185bbf1dd1a/users/3fc92381d6f546abac4e8e84708df4a1/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:03.027762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed2091-3958-48a6-b8ae-caab54b5351c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:03.028050 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed2091-3958-48a6-b8ae-caab54b5351c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:03.028320 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed2091-3958-48a6-b8ae-caab54b5351c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:03.056272 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3fed2091-3958-48a6-b8ae-caab54b5351c None None] MFA Rules not processed for user `3fc92381d6f546abac4e8e84708df4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:03.057501 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3fed2091-3958-48a6-b8ae-caab54b5351c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:03.128537 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1374/2786] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:03.139308 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1399a-52fc-45ef-87d3-33c53fb55dcf admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:03.141156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-47a1399a-52fc-45ef-87d3-33c53fb55dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:03.200779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1399a-52fc-45ef-87d3-33c53fb55dcf admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:03.206076 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82b500-f741-4779-a8ac-9766cd490c31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:03.206674 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82b500-f741-4779-a8ac-9766cd490c31 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:03.206948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82b500-f741-4779-a8ac-9766cd490c31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:03.207209 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82b500-f741-4779-a8ac-9766cd490c31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:03.264524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82b500-f741-4779-a8ac-9766cd490c31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:03.265964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82b500-f741-4779-a8ac-9766cd490c31 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:03.275488 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1413/2787] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:41:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:41:11.005628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed2091-3958-48a6-b8ae-caab54b5351c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:11.015161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed2091-3958-48a6-b8ae-caab54b5351c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:11.018680 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afc5fc-2cb1-4c82-a8e3-f0de023af840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:11.019270 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afc5fc-2cb1-4c82-a8e3-f0de023af840 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:11.019546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afc5fc-2cb1-4c82-a8e3-f0de023af840 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:11.019816 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afc5fc-2cb1-4c82-a8e3-f0de023af840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:11.028903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afc5fc-2cb1-4c82-a8e3-f0de023af840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:11.030520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afc5fc-2cb1-4c82-a8e3-f0de023af840 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:11.040810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1375/2788] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:41:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:41:13.271499 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=71,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:21.036308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=69 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:37.111658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af2920-e253-4ca1-bfdd-9dafc699fe2f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:37.112117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af2920-e253-4ca1-bfdd-9dafc699fe2f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:37.112117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af2920-e253-4ca1-bfdd-9dafc699fe2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:37.133149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-a8af2920-e253-4ca1-bfdd-9dafc699fe2f None None] MFA Rules not processed for user `eccbf92faf2b4bad8746a2a48f5a6226`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:37.134359 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a8af2920-e253-4ca1-bfdd-9dafc699fe2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:37.198969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1414/2789] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:47.190481 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=25 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:47.314092 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afc5fc-2cb1-4c82-a8e3-f0de023af840 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:47.374708 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afc5fc-2cb1-4c82-a8e3-f0de023af840 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:47.379147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:47.379786 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:47.380062 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:47.380331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] PATH_INFO: `/v3/users/722a5b5ba7b144fdaa47eec75d6d9fc5` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:47.381958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=722a5b5ba7b144fdaa47eec75d6d9fc5)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:47.383877 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:47.430429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '722a5b5ba7b144fdaa47eec75d6d9fc5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:47.437325 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '722a5b5ba7b144fdaa47eec75d6d9fc5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:47.437986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '722a5b5ba7b144fdaa47eec75d6d9fc5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:47.444894 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '722a5b5ba7b144fdaa47eec75d6d9fc5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:47.456496 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '722a5b5ba7b144fdaa47eec75d6d9fc5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:47.462768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1376/2790] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:41:47 2022] DELETE /identity/v3/users/722a5b5ba7b144fdaa47eec75d6d9fc5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:47.467278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34599a53-59c7-47b4-81e2-4bb91f460272 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:47.467567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34599a53-59c7-47b4-81e2-4bb91f460272 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:47.467845 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34599a53-59c7-47b4-81e2-4bb91f460272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:47.486305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-34599a53-59c7-47b4-81e2-4bb91f460272 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:47.487490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-34599a53-59c7-47b4-81e2-4bb91f460272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:47.544667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1415/2791] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:47.555296 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:47.556741 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:47.612103 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0e18c-24dc-40af-994f-deadd8d781bb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:47.616308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bf3bc-008d-4c80-b8b1-a087b9f0e28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:47.616938 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bf3bc-008d-4c80-b8b1-a087b9f0e28a service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:47.617216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bf3bc-008d-4c80-b8b1-a087b9f0e28a service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:47.617486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bf3bc-008d-4c80-b8b1-a087b9f0e28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:47.626349 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bf3bc-008d-4c80-b8b1-a087b9f0e28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:47.627714 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bf3bc-008d-4c80-b8b1-a087b9f0e28a service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:47.637628 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1377/2792] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:41:47.905211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34599a53-59c7-47b4-81e2-4bb91f460272 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:47.906635 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-34599a53-59c7-47b4-81e2-4bb91f460272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:47.920393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34599a53-59c7-47b4-81e2-4bb91f460272 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:47.925167 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:47.925831 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:47.926135 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:47.926469 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] PATH_INFO: `/v3/projects/543f256d05a440bfa15dac02b5c0a40e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:47.927632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=543f256d05a440bfa15dac02b5c0a40e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:47.930211 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:47.976217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '543f256d05a440bfa15dac02b5c0a40e', 'request_id': 'req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:47.981887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '543f256d05a440bfa15dac02b5c0a40e', 'request_id': 'req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:47.987956 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '543f256d05a440bfa15dac02b5c0a40e', 'request_id': 'req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.000388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] The token cache is being invalidate because project 543f256d05a440bfa15dac02b5c0a40e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:41:48.000693 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.001300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.002766 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1416/2793] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:41:47 2022] DELETE /identity/v3/projects/543f256d05a440bfa15dac02b5c0a40e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:48.006808 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bf3bc-008d-4c80-b8b1-a087b9f0e28a service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:48.016224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bf3bc-008d-4c80-b8b1-a087b9f0e28a service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:48.033951 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:48.074345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd23910-8231-4c3c-877c-2c1a26c666fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:48.074972 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd23910-8231-4c3c-877c-2c1a26c666fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:48.075249 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd23910-8231-4c3c-877c-2c1a26c666fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:48.075522 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd23910-8231-4c3c-877c-2c1a26c666fe admin admin] PATH_INFO: `/v3/users/3fc92381d6f546abac4e8e84708df4a1` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:48.076670 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd23910-8231-4c3c-877c-2c1a26c666fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc92381d6f546abac4e8e84708df4a1)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:48.078584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd23910-8231-4c3c-877c-2c1a26c666fe admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:48.133489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-acd23910-8231-4c3c-877c-2c1a26c666fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc92381d6f546abac4e8e84708df4a1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.140532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-acd23910-8231-4c3c-877c-2c1a26c666fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc92381d6f546abac4e8e84708df4a1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.141204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-acd23910-8231-4c3c-877c-2c1a26c666fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc92381d6f546abac4e8e84708df4a1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.149435 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-acd23910-8231-4c3c-877c-2c1a26c666fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc92381d6f546abac4e8e84708df4a1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.161959 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-acd23910-8231-4c3c-877c-2c1a26c666fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc92381d6f546abac4e8e84708df4a1'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.168491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1378/2794] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:41:48 2022] DELETE /identity/v3/users/3fc92381d6f546abac4e8e84708df4a1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:48.469510 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:48.471139 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:48.484169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a0b98-b18c-4fa1-819f-32d84113b9c8 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:48.540451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:48.541079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:48.541355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:48.541625 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] PATH_INFO: `/v3/projects/2ef4a8df3c874dd9b53b4185bbf1dd1a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:48.542722 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef4a8df3c874dd9b53b4185bbf1dd1a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:48.544642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:48.574957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:48.592446 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef4a8df3c874dd9b53b4185bbf1dd1a', 'request_id': 'req-2b0414e4-6514-43fd-be3b-1cc4bd551289'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.597800 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef4a8df3c874dd9b53b4185bbf1dd1a', 'request_id': 'req-2b0414e4-6514-43fd-be3b-1cc4bd551289'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.603740 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef4a8df3c874dd9b53b4185bbf1dd1a', 'request_id': 'req-2b0414e4-6514-43fd-be3b-1cc4bd551289'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.616059 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] The token cache is being invalidate because project 2ef4a8df3c874dd9b53b4185bbf1dd1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:41:48.616343 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.616969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-2b0414e4-6514-43fd-be3b-1cc4bd551289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:48.618413 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1417/2795] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:41:48 2022] DELETE /identity/v3/projects/2ef4a8df3c874dd9b53b4185bbf1dd1a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:41:48.623210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fdba-cd23-46d0-a81a-458faa4b135c None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:48.623685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fdba-cd23-46d0-a81a-458faa4b135c None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:48.623793 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388fdba-cd23-46d0-a81a-458faa4b135c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:48.643963 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-3388fdba-cd23-46d0-a81a-458faa4b135c None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:48.645171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-3388fdba-cd23-46d0-a81a-458faa4b135c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:48.702970 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1379/2796] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:48.710724 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39211590-1ae4-4580-aad8-4e19b8b40863 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:48.711012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39211590-1ae4-4580-aad8-4e19b8b40863 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:48.711281 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39211590-1ae4-4580-aad8-4e19b8b40863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:48.735926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-39211590-1ae4-4580-aad8-4e19b8b40863 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:48.737340 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-39211590-1ae4-4580-aad8-4e19b8b40863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:48.746431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1418/2797] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:48.750831 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388fdba-cd23-46d0-a81a-458faa4b135c None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:48.760748 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388fdba-cd23-46d0-a81a-458faa4b135c None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:48.765363 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fd6c5-c8dd-4f89-9b15-1215ba4b13a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:48.765957 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fd6c5-c8dd-4f89-9b15-1215ba4b13a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:48.766235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fd6c5-c8dd-4f89-9b15-1215ba4b13a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:48.766504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fd6c5-c8dd-4f89-9b15-1215ba4b13a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:48.767173 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fd6c5-c8dd-4f89-9b15-1215ba4b13a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:41:48.767470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fd6c5-c8dd-4f89-9b15-1215ba4b13a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:48.769108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fd6c5-c8dd-4f89-9b15-1215ba4b13a4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:48.780843 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1380/2798] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:41:48.785141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39211590-1ae4-4580-aad8-4e19b8b40863 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:48.794961 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39211590-1ae4-4580-aad8-4e19b8b40863 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:48.799520 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c86dd4-4299-4a79-be51-aa03e1275c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:48.800101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c86dd4-4299-4a79-be51-aa03e1275c29 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:48.800392 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c86dd4-4299-4a79-be51-aa03e1275c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:48.800664 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c86dd4-4299-4a79-be51-aa03e1275c29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:48.801289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c86dd4-4299-4a79-be51-aa03e1275c29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:48.803806 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c86dd4-4299-4a79-be51-aa03e1275c29 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:48.829143 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1419/2799] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:41:48 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:41:48.833216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fd6c5-c8dd-4f89-9b15-1215ba4b13a4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:48.843047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fd6c5-c8dd-4f89-9b15-1215ba4b13a4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:48.869077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:48.900496 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efa0c2-1ef0-428d-bffc-c3d02073a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:48.901090 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efa0c2-1ef0-428d-bffc-c3d02073a6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:48.901392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efa0c2-1ef0-428d-bffc-c3d02073a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:48.901659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efa0c2-1ef0-428d-bffc-c3d02073a6a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:48.902285 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efa0c2-1ef0-428d-bffc-c3d02073a6a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:48.904178 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efa0c2-1ef0-428d-bffc-c3d02073a6a2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:48.925068 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1381/2800] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:41:48 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:41:48.930096 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c86dd4-4299-4a79-be51-aa03e1275c29 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:48.940011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c86dd4-4299-4a79-be51-aa03e1275c29 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:48.944602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b57c7-43b7-4299-9d8b-0e0713311b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:48.945214 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b57c7-43b7-4299-9d8b-0e0713311b73 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:48.945483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b57c7-43b7-4299-9d8b-0e0713311b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:48.945746 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b57c7-43b7-4299-9d8b-0e0713311b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:48.946434 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b57c7-43b7-4299-9d8b-0e0713311b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:48.947990 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b57c7-43b7-4299-9d8b-0e0713311b73 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:48.958012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1420/2801] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:41:48.961755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efa0c2-1ef0-428d-bffc-c3d02073a6a2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:48.971461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efa0c2-1ef0-428d-bffc-c3d02073a6a2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:48.976502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d120b4-8d1f-4681-9d2b-6176dc5d3c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:48.977079 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d120b4-8d1f-4681-9d2b-6176dc5d3c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:48.977357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d120b4-8d1f-4681-9d2b-6176dc5d3c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:48.977626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d120b4-8d1f-4681-9d2b-6176dc5d3c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:48.978250 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d120b4-8d1f-4681-9d2b-6176dc5d3c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:48.979826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d120b4-8d1f-4681-9d2b-6176dc5d3c63 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:48.989429 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1382/2802] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:41:48.993803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b57c7-43b7-4299-9d8b-0e0713311b73 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:49.003350 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b57c7-43b7-4299-9d8b-0e0713311b73 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:49.007804 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e8678-f4e9-497d-b32f-272187669e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:49.008449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e8678-f4e9-497d-b32f-272187669e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:49.008723 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e8678-f4e9-497d-b32f-272187669e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:49.008988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e8678-f4e9-497d-b32f-272187669e1e admin admin] PATH_INFO: `/v3/projects/dda429eac10a4f2ab987f98f4fac9dff/users/99f5bbc7f044440fa38dfb7983d34a51/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:49.023364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e8678-f4e9-497d-b32f-272187669e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda429eac10a4f2ab987f98f4fac9dff, user_id=99f5bbc7f044440fa38dfb7983d34a51, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:49.026056 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e8678-f4e9-497d-b32f-272187669e1e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:49.039286 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1421/2803] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:41:48 2022] PUT /identity/v3/projects/dda429eac10a4f2ab987f98f4fac9dff/users/99f5bbc7f044440fa38dfb7983d34a51/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:49.050623 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedb5c1-d80e-4835-9d4c-6a97de12bd10 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:49.050905 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedb5c1-d80e-4835-9d4c-6a97de12bd10 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:49.051203 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedb5c1-d80e-4835-9d4c-6a97de12bd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:49.079671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-6fedb5c1-d80e-4835-9d4c-6a97de12bd10 None None] MFA Rules not processed for user `99f5bbc7f044440fa38dfb7983d34a51`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:49.080908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6fedb5c1-d80e-4835-9d4c-6a97de12bd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:49.146452 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1383/2804] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:49.151569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e8678-f4e9-497d-b32f-272187669e1e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:49.215364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e8678-f4e9-497d-b32f-272187669e1e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:49.220137 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0fd71-9874-4adc-9cdd-0a17b06ffa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:49.220805 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fd71-9874-4adc-9cdd-0a17b06ffa2f admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:49.221087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fd71-9874-4adc-9cdd-0a17b06ffa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:49.221358 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fd71-9874-4adc-9cdd-0a17b06ffa2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:49.222018 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0fd71-9874-4adc-9cdd-0a17b06ffa2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:49.223927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0fd71-9874-4adc-9cdd-0a17b06ffa2f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:49.256134 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1422/2805] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:41:49 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:41:49.274158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedb5c1-d80e-4835-9d4c-6a97de12bd10 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:49.287042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedb5c1-d80e-4835-9d4c-6a97de12bd10 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:49.354779 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301af26-9e9c-4638-a326-be167c5d9c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:49.356101 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301af26-9e9c-4638-a326-be167c5d9c28 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:49.356405 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301af26-9e9c-4638-a326-be167c5d9c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:49.356680 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301af26-9e9c-4638-a326-be167c5d9c28 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:49.357314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301af26-9e9c-4638-a326-be167c5d9c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:49.359237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301af26-9e9c-4638-a326-be167c5d9c28 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:49.380469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1384/2806] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:41:49 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:41:49.384472 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0fd71-9874-4adc-9cdd-0a17b06ffa2f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:49.396048 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0fd71-9874-4adc-9cdd-0a17b06ffa2f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:49.407713 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed9160-774e-4368-bf17-79292918930b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:49.408411 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed9160-774e-4368-bf17-79292918930b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:49.408688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed9160-774e-4368-bf17-79292918930b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:49.408985 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed9160-774e-4368-bf17-79292918930b admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:49.409630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed9160-774e-4368-bf17-79292918930b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:49.411473 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed9160-774e-4368-bf17-79292918930b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:49.425623 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1423/2807] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 19:41:49.429826 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301af26-9e9c-4638-a326-be167c5d9c28 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:49.439914 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301af26-9e9c-4638-a326-be167c5d9c28 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:49.445141 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6f1f8-2df3-404b-aa7c-4ba17ec40547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:49.445784 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6f1f8-2df3-404b-aa7c-4ba17ec40547 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:49.446060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6f1f8-2df3-404b-aa7c-4ba17ec40547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:49.446332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6f1f8-2df3-404b-aa7c-4ba17ec40547 admin admin] PATH_INFO: `/v3/projects/3622d83e4329409e90f4609acc70910e/users/a560b13efaa2461da7475027d562d5b9/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:49.463330 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6f1f8-2df3-404b-aa7c-4ba17ec40547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3622d83e4329409e90f4609acc70910e, user_id=a560b13efaa2461da7475027d562d5b9, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:49.466314 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6f1f8-2df3-404b-aa7c-4ba17ec40547 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:49.496543 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1385/2808] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:41:49 2022] PUT /identity/v3/projects/3622d83e4329409e90f4609acc70910e/users/a560b13efaa2461da7475027d562d5b9/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:49.500775 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed9160-774e-4368-bf17-79292918930b admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:49.518596 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed9160-774e-4368-bf17-79292918930b admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:49.540859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:49.581302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86cfe83-1099-43ba-abcc-b386faad836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:49.582016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cfe83-1099-43ba-abcc-b386faad836b admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:49.582299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cfe83-1099-43ba-abcc-b386faad836b admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:49.582572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cfe83-1099-43ba-abcc-b386faad836b admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:49.583212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86cfe83-1099-43ba-abcc-b386faad836b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:49.585376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86cfe83-1099-43ba-abcc-b386faad836b admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:49.595271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1424/2809] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:41:49.599500 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6f1f8-2df3-404b-aa7c-4ba17ec40547 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:49.605042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:49.611717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6f1f8-2df3-404b-aa7c-4ba17ec40547 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:49.616666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c326cf-a15d-4503-b0da-530178fd4a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:49.617308 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c326cf-a15d-4503-b0da-530178fd4a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:49.617580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c326cf-a15d-4503-b0da-530178fd4a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:49.617872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c326cf-a15d-4503-b0da-530178fd4a04 admin admin] PATH_INFO: `/v3/projects/3622d83e4329409e90f4609acc70910e/users/a560b13efaa2461da7475027d562d5b9/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:49.619755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c326cf-a15d-4503-b0da-530178fd4a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3622d83e4329409e90f4609acc70910e, user_id=a560b13efaa2461da7475027d562d5b9, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:49.622432 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c326cf-a15d-4503-b0da-530178fd4a04 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:49.634048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1386/2810] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:41:49 2022] PUT /identity/v3/projects/3622d83e4329409e90f4609acc70910e/users/a560b13efaa2461da7475027d562d5b9/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:49.644071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1518423-e43a-422b-ad8a-3c0f410c124b None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:49.644385 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1518423-e43a-422b-ad8a-3c0f410c124b None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:49.644665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1518423-e43a-422b-ad8a-3c0f410c124b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:49.673416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-b1518423-e43a-422b-ad8a-3c0f410c124b None None] MFA Rules not processed for user `a560b13efaa2461da7475027d562d5b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:49.674613 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-b1518423-e43a-422b-ad8a-3c0f410c124b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:49.743172 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1425/2811] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:49.755928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c326cf-a15d-4503-b0da-530178fd4a04 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:49.758139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-33c326cf-a15d-4503-b0da-530178fd4a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:49.818390 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c326cf-a15d-4503-b0da-530178fd4a04 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:49.822634 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213c293-c0f3-4de1-9824-5cde089c0e42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:49.823245 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213c293-c0f3-4de1-9824-5cde089c0e42 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:49.823671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213c293-c0f3-4de1-9824-5cde089c0e42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:49.823896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213c293-c0f3-4de1-9824-5cde089c0e42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:49.833943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213c293-c0f3-4de1-9824-5cde089c0e42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:49.835382 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213c293-c0f3-4de1-9824-5cde089c0e42 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:49.845112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1387/2812] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:41:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:41:49.912580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1518423-e43a-422b-ad8a-3c0f410c124b None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:49.974837 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1518423-e43a-422b-ad8a-3c0f410c124b None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:49.979565 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:49.980289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:49.980583 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:49.980902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] PATH_INFO: `/v3/users/99f5bbc7f044440fa38dfb7983d34a51` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:49.982448 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f5bbc7f044440fa38dfb7983d34a51)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:49.984524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:50.031085 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f5bbc7f044440fa38dfb7983d34a51'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.032189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:50.039867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f5bbc7f044440fa38dfb7983d34a51'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.040538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f5bbc7f044440fa38dfb7983d34a51'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.047562 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f5bbc7f044440fa38dfb7983d34a51'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.060067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f5bbc7f044440fa38dfb7983d34a51'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.065866 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1426/2813] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:41:49 2022] DELETE /identity/v3/users/99f5bbc7f044440fa38dfb7983d34a51 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:50.071017 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288be9-6c96-4847-85f5-b83f55aa2eb0 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:50.071310 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288be9-6c96-4847-85f5-b83f55aa2eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:50.071573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288be9-6c96-4847-85f5-b83f55aa2eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:50.089996 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-81288be9-6c96-4847-85f5-b83f55aa2eb0 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:50.091224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-81288be9-6c96-4847-85f5-b83f55aa2eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:50.148387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1388/2814] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:50.158029 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:50.159445 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:50.215914 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f92466-6c9f-43db-9b09-7a942e97706e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:50.219554 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687c029-dd32-4e56-9bef-4ec638c29eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:50.220156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687c029-dd32-4e56-9bef-4ec638c29eaf service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:50.220462 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687c029-dd32-4e56-9bef-4ec638c29eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:50.220733 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687c029-dd32-4e56-9bef-4ec638c29eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:50.229945 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0687c029-dd32-4e56-9bef-4ec638c29eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:50.231362 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0687c029-dd32-4e56-9bef-4ec638c29eaf service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:50.241224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1427/2815] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:41:50.499802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81288be9-6c96-4847-85f5-b83f55aa2eb0 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:50.501295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-81288be9-6c96-4847-85f5-b83f55aa2eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:50.513038 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81288be9-6c96-4847-85f5-b83f55aa2eb0 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:50.517568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:50.518192 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:50.518467 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:50.518739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] PATH_INFO: `/v3/projects/dda429eac10a4f2ab987f98f4fac9dff` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:50.519875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda429eac10a4f2ab987f98f4fac9dff)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:50.521982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:50.570966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda429eac10a4f2ab987f98f4fac9dff', 'request_id': 'req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.576353 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda429eac10a4f2ab987f98f4fac9dff', 'request_id': 'req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.582417 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda429eac10a4f2ab987f98f4fac9dff', 'request_id': 'req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.595237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] The token cache is being invalidate because project dda429eac10a4f2ab987f98f4fac9dff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:41:50.595518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.596124 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.597581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1389/2816] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:41:50 2022] DELETE /identity/v3/projects/dda429eac10a4f2ab987f98f4fac9dff => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:50.602156 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687c029-dd32-4e56-9bef-4ec638c29eaf service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:50.611490 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687c029-dd32-4e56-9bef-4ec638c29eaf service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:50.666774 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87daa7-a19b-4f45-bf68-e3a1fe247f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:50.667391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87daa7-a19b-4f45-bf68-e3a1fe247f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:50.667666 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87daa7-a19b-4f45-bf68-e3a1fe247f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:50.667927 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87daa7-a19b-4f45-bf68-e3a1fe247f7c admin admin] PATH_INFO: `/v3/users/a560b13efaa2461da7475027d562d5b9` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:50.668992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87daa7-a19b-4f45-bf68-e3a1fe247f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a560b13efaa2461da7475027d562d5b9)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:50.670883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87daa7-a19b-4f45-bf68-e3a1fe247f7c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:50.719763 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3d87daa7-a19b-4f45-bf68-e3a1fe247f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a560b13efaa2461da7475027d562d5b9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.726656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3d87daa7-a19b-4f45-bf68-e3a1fe247f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a560b13efaa2461da7475027d562d5b9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.727317 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3d87daa7-a19b-4f45-bf68-e3a1fe247f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a560b13efaa2461da7475027d562d5b9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.735016 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3d87daa7-a19b-4f45-bf68-e3a1fe247f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a560b13efaa2461da7475027d562d5b9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.741429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:50.752225 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3d87daa7-a19b-4f45-bf68-e3a1fe247f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a560b13efaa2461da7475027d562d5b9'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:50.759140 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1428/2817] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:41:50 2022] DELETE /identity/v3/users/a560b13efaa2461da7475027d562d5b9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:51.268773 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:51.270235 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:51.284423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fafb9f-44ad-4e2d-a1e1-355cf3d5715e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:51.298511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:51.345237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:51.345864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:51.346143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:51.346413 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] PATH_INFO: `/v3/projects/3622d83e4329409e90f4609acc70910e` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:51.347511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3622d83e4329409e90f4609acc70910e)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:51.349461 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:51.394851 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3622d83e4329409e90f4609acc70910e', 'request_id': 'req-e22a557c-2edf-4d9e-98a9-7cd0db48f261'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:51.400757 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3622d83e4329409e90f4609acc70910e', 'request_id': 'req-e22a557c-2edf-4d9e-98a9-7cd0db48f261'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:51.406890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3622d83e4329409e90f4609acc70910e', 'request_id': 'req-e22a557c-2edf-4d9e-98a9-7cd0db48f261'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:51.419899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] The token cache is being invalidate because project 3622d83e4329409e90f4609acc70910e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:41:51.420185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:51.420855 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e22a557c-2edf-4d9e-98a9-7cd0db48f261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:41:51.422328 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1390/2818] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:41:51 2022] DELETE /identity/v3/projects/3622d83e4329409e90f4609acc70910e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:51.427284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df491173-b8be-48b5-8570-372a3fbf72ad None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:51.427569 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df491173-b8be-48b5-8570-372a3fbf72ad None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:51.427847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df491173-b8be-48b5-8570-372a3fbf72ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:51.447351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-df491173-b8be-48b5-8570-372a3fbf72ad None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:51.448521 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-df491173-b8be-48b5-8570-372a3fbf72ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:51.505584 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1429/2819] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:51.513367 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92506636-a3b3-4dc6-9efd-43219e983be9 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:51.513662 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92506636-a3b3-4dc6-9efd-43219e983be9 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:51.513939 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92506636-a3b3-4dc6-9efd-43219e983be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:51.533491 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-92506636-a3b3-4dc6-9efd-43219e983be9 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:51.534685 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-92506636-a3b3-4dc6-9efd-43219e983be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:51.543447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1391/2820] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:51.547614 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df491173-b8be-48b5-8570-372a3fbf72ad None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:51.557325 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df491173-b8be-48b5-8570-372a3fbf72ad None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:51.561934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b0703-9f6c-4716-96b8-491636ba7fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:51.562532 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b0703-9f6c-4716-96b8-491636ba7fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:51.562808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b0703-9f6c-4716-96b8-491636ba7fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:51.563080 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b0703-9f6c-4716-96b8-491636ba7fdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:51.563721 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b0703-9f6c-4716-96b8-491636ba7fdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:41:51.564019 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b0703-9f6c-4716-96b8-491636ba7fdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:51.565632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b0703-9f6c-4716-96b8-491636ba7fdd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:51.577482 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1430/2821] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:41:51.581600 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92506636-a3b3-4dc6-9efd-43219e983be9 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:51.591518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92506636-a3b3-4dc6-9efd-43219e983be9 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:51.596163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e615e9f-2178-4593-9434-4ab12f35005e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:51.596788 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e615e9f-2178-4593-9434-4ab12f35005e admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:51.597067 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e615e9f-2178-4593-9434-4ab12f35005e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:51.597340 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e615e9f-2178-4593-9434-4ab12f35005e admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:51.597973 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e615e9f-2178-4593-9434-4ab12f35005e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:51.600575 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e615e9f-2178-4593-9434-4ab12f35005e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:51.627237 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1392/2822] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:41:51 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:41:51.631390 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b0703-9f6c-4716-96b8-491636ba7fdd admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:51.641451 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b0703-9f6c-4716-96b8-491636ba7fdd admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:51.696907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115fb6b-e0b1-48a1-9604-7d9faadbe6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:51.697556 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fb6b-e0b1-48a1-9604-7d9faadbe6e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:51.697787 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fb6b-e0b1-48a1-9604-7d9faadbe6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:51.698057 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115fb6b-e0b1-48a1-9604-7d9faadbe6e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:51.698735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115fb6b-e0b1-48a1-9604-7d9faadbe6e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:51.700767 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115fb6b-e0b1-48a1-9604-7d9faadbe6e9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:51.722001 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1431/2823] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:41:51 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:41:51.726715 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e615e9f-2178-4593-9434-4ab12f35005e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:51.737160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e615e9f-2178-4593-9434-4ab12f35005e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:51.741875 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dbc03-a1c7-415d-ba93-154fde54e51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:51.742454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dbc03-a1c7-415d-ba93-154fde54e51e admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:51.742727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dbc03-a1c7-415d-ba93-154fde54e51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:51.742991 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dbc03-a1c7-415d-ba93-154fde54e51e admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:51.743617 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dbc03-a1c7-415d-ba93-154fde54e51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:51.745226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dbc03-a1c7-415d-ba93-154fde54e51e admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:51.754961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1393/2824] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:41:51.759678 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115fb6b-e0b1-48a1-9604-7d9faadbe6e9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:51.769252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115fb6b-e0b1-48a1-9604-7d9faadbe6e9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:51.774311 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514daab4-ee9d-48c7-bfdf-0c7a3985f216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:51.774902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514daab4-ee9d-48c7-bfdf-0c7a3985f216 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:51.775163 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514daab4-ee9d-48c7-bfdf-0c7a3985f216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:51.775429 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514daab4-ee9d-48c7-bfdf-0c7a3985f216 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:51.776058 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514daab4-ee9d-48c7-bfdf-0c7a3985f216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:51.777655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514daab4-ee9d-48c7-bfdf-0c7a3985f216 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:51.788300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1432/2825] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:41:51.792468 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dbc03-a1c7-415d-ba93-154fde54e51e admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:51.802046 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dbc03-a1c7-415d-ba93-154fde54e51e admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:51.806602 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9440b27-d894-433e-9890-a7091a62cec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:51.807230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9440b27-d894-433e-9890-a7091a62cec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:51.807509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9440b27-d894-433e-9890-a7091a62cec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:51.807780 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9440b27-d894-433e-9890-a7091a62cec0 admin admin] PATH_INFO: `/v3/projects/cfad18f51de14ecd91e1cd94c0fdb614/users/f8d28ff7471948c4b1b55d467e345ce8/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:51.822402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9440b27-d894-433e-9890-a7091a62cec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfad18f51de14ecd91e1cd94c0fdb614, user_id=f8d28ff7471948c4b1b55d467e345ce8, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:51.825077 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9440b27-d894-433e-9890-a7091a62cec0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:51.838681 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1394/2826] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:41:51 2022] PUT /identity/v3/projects/cfad18f51de14ecd91e1cd94c0fdb614/users/f8d28ff7471948c4b1b55d467e345ce8/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:51.849773 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96652eb6-2e1b-4979-a131-6917fa524ce9 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:51.850055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96652eb6-2e1b-4979-a131-6917fa524ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:51.850326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96652eb6-2e1b-4979-a131-6917fa524ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:51.877430 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-96652eb6-2e1b-4979-a131-6917fa524ce9 None None] MFA Rules not processed for user `f8d28ff7471948c4b1b55d467e345ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:51.878707 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-96652eb6-2e1b-4979-a131-6917fa524ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:51.920622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:51.948498 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1433/2827] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:51.952830 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9440b27-d894-433e-9890-a7091a62cec0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:51.998888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:52.017776 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9440b27-d894-433e-9890-a7091a62cec0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:52.023445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd036f-1c7c-4697-bec7-9b570513f822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:52.024048 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd036f-1c7c-4697-bec7-9b570513f822 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:52.024594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd036f-1c7c-4697-bec7-9b570513f822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:52.024594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd036f-1c7c-4697-bec7-9b570513f822 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:52.025230 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd036f-1c7c-4697-bec7-9b570513f822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:52.027125 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd036f-1c7c-4697-bec7-9b570513f822 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:52.053538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1395/2828] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:41:51 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:41:52.057647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96652eb6-2e1b-4979-a131-6917fa524ce9 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:52.067896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96652eb6-2e1b-4979-a131-6917fa524ce9 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:52.121575 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d043e9-0c0b-453c-924e-626ce36e6e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:52.122166 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d043e9-0c0b-453c-924e-626ce36e6e0c admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:52.122438 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d043e9-0c0b-453c-924e-626ce36e6e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:52.122701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d043e9-0c0b-453c-924e-626ce36e6e0c admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:52.123306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d043e9-0c0b-453c-924e-626ce36e6e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:52.125212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d043e9-0c0b-453c-924e-626ce36e6e0c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:52.146089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1434/2829] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:41:52 2022] POST /identity/v3/users => generated 303 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:41:52.150063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd036f-1c7c-4697-bec7-9b570513f822 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:52.160098 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd036f-1c7c-4697-bec7-9b570513f822 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:52.164876 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5d9cd-923b-4dbf-a79a-f6c7cec29bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:52.165457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5d9cd-923b-4dbf-a79a-f6c7cec29bba admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:52.165735 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5d9cd-923b-4dbf-a79a-f6c7cec29bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:52.166002 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5d9cd-923b-4dbf-a79a-f6c7cec29bba admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:52.166632 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5d9cd-923b-4dbf-a79a-f6c7cec29bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:52.168204 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5d9cd-923b-4dbf-a79a-f6c7cec29bba admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:52.178890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1396/2830] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:41:52.182859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d043e9-0c0b-453c-924e-626ce36e6e0c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:52.192553 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d043e9-0c0b-453c-924e-626ce36e6e0c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:52.197727 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69e042-e8a9-47dc-99d0-4e6e79083413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:52.198363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69e042-e8a9-47dc-99d0-4e6e79083413 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:52.198836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69e042-e8a9-47dc-99d0-4e6e79083413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:52.199012 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69e042-e8a9-47dc-99d0-4e6e79083413 admin admin] PATH_INFO: `/v3/projects/ccefb3d6dc17499ea1541f6058cc8202/users/3d0b8866017d49f4814b33f7a665bcc7/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:52.215938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69e042-e8a9-47dc-99d0-4e6e79083413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccefb3d6dc17499ea1541f6058cc8202, user_id=3d0b8866017d49f4814b33f7a665bcc7, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:52.218595 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69e042-e8a9-47dc-99d0-4e6e79083413 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:52.244068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1435/2831] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:41:52 2022] PUT /identity/v3/projects/ccefb3d6dc17499ea1541f6058cc8202/users/3d0b8866017d49f4814b33f7a665bcc7/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:52.248058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5d9cd-923b-4dbf-a79a-f6c7cec29bba admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:52.258506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5d9cd-923b-4dbf-a79a-f6c7cec29bba admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:52.313030 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8f92b-a1ca-4b24-949e-fab5f8d355d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:52.313671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8f92b-a1ca-4b24-949e-fab5f8d355d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:52.313888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8f92b-a1ca-4b24-949e-fab5f8d355d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:52.314153 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8f92b-a1ca-4b24-949e-fab5f8d355d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:52.314778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8f92b-a1ca-4b24-949e-fab5f8d355d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:52.316347 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8f92b-a1ca-4b24-949e-fab5f8d355d1 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:52.326456 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1397/2832] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:41:52.330637 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69e042-e8a9-47dc-99d0-4e6e79083413 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:52.340677 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69e042-e8a9-47dc-99d0-4e6e79083413 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:52.345313 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359402b8-d6fc-49d7-91b5-491b4914ff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:52.345953 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359402b8-d6fc-49d7-91b5-491b4914ff7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:52.346230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359402b8-d6fc-49d7-91b5-491b4914ff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:52.346504 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359402b8-d6fc-49d7-91b5-491b4914ff7f admin admin] PATH_INFO: `/v3/projects/ccefb3d6dc17499ea1541f6058cc8202/users/3d0b8866017d49f4814b33f7a665bcc7/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:52.348431 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359402b8-d6fc-49d7-91b5-491b4914ff7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccefb3d6dc17499ea1541f6058cc8202, user_id=3d0b8866017d49f4814b33f7a665bcc7, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:52.351082 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359402b8-d6fc-49d7-91b5-491b4914ff7f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:52.360957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1436/2833] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:41:52 2022] PUT /identity/v3/projects/ccefb3d6dc17499ea1541f6058cc8202/users/3d0b8866017d49f4814b33f7a665bcc7/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:41:52.371355 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f96ce-0681-4896-a659-d04306b8c06d None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:52.371705 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f96ce-0681-4896-a659-d04306b8c06d None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:52.372040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f96ce-0681-4896-a659-d04306b8c06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:52.400392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a29f96ce-0681-4896-a659-d04306b8c06d None None] MFA Rules not processed for user `3d0b8866017d49f4814b33f7a665bcc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:41:52.401597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a29f96ce-0681-4896-a659-d04306b8c06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:52.468358 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1398/2834] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:41:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:41:52.479266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359402b8-d6fc-49d7-91b5-491b4914ff7f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:52.481071 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-359402b8-d6fc-49d7-91b5-491b4914ff7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:52.536980 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359402b8-d6fc-49d7-91b5-491b4914ff7f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:52.541141 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce13a02-bd24-47a8-8936-8dbd5df1da23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:52.541735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce13a02-bd24-47a8-8936-8dbd5df1da23 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:52.542011 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce13a02-bd24-47a8-8936-8dbd5df1da23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:52.542278 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce13a02-bd24-47a8-8936-8dbd5df1da23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:52.550934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce13a02-bd24-47a8-8936-8dbd5df1da23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:52.552351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce13a02-bd24-47a8-8936-8dbd5df1da23 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:52.571926 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1437/2835] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:41:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:41:52.620160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f96ce-0681-4896-a659-d04306b8c06d None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:41:52.621576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a29f96ce-0681-4896-a659-d04306b8c06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:41:52.633258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f96ce-0681-4896-a659-d04306b8c06d None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:41:52.636785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d6110-b01b-462d-ad10-258aa8697bf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:41:52.637383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d6110-b01b-462d-ad10-258aa8697bf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:41:52.637642 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d6110-b01b-462d-ad10-258aa8697bf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:41:52.637907 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d6110-b01b-462d-ad10-258aa8697bf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:41:52.670437 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:41:52.694535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d6110-b01b-462d-ad10-258aa8697bf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:41:52.695945 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d6110-b01b-462d-ad10-258aa8697bf7 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:41:52.704621 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1399/2836] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:41:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:42:00.200195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce13a02-bd24-47a8-8936-8dbd5df1da23 service cinder] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:00.242911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:00.272216 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce13a02-bd24-47a8-8936-8dbd5df1da23 service cinder] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:00.277466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:00.278089 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:00.278363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:00.278630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] PATH_INFO: `/v3/users/f8d28ff7471948c4b1b55d467e345ce8` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:00.279735 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d28ff7471948c4b1b55d467e345ce8)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:00.281654 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:00.326594 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d28ff7471948c4b1b55d467e345ce8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:00.333222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d28ff7471948c4b1b55d467e345ce8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:00.333861 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d28ff7471948c4b1b55d467e345ce8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:00.341244 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d28ff7471948c4b1b55d467e345ce8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:00.353769 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d28ff7471948c4b1b55d467e345ce8'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:00.359425 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1438/2837] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:42:00 2022] DELETE /identity/v3/users/f8d28ff7471948c4b1b55d467e345ce8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:42:00.363896 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9b408-3399-42a1-803a-1b20aaa330a6 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:00.364392 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9b408-3399-42a1-803a-1b20aaa330a6 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:00.364496 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9b408-3399-42a1-803a-1b20aaa330a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:00.383624 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-76e9b408-3399-42a1-803a-1b20aaa330a6 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:00.384850 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-76e9b408-3399-42a1-803a-1b20aaa330a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:00.442163 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1400/2838] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:00.452060 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:00.453541 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:00.509109 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec9462-f55b-4b03-98ff-1efd1de9e65e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:00.512686 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03ed21-6591-491d-9489-7fa69369f2ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:00.513273 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03ed21-6591-491d-9489-7fa69369f2ae service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:00.513545 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03ed21-6591-491d-9489-7fa69369f2ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:00.513811 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03ed21-6591-491d-9489-7fa69369f2ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:00.523602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03ed21-6591-491d-9489-7fa69369f2ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:00.525097 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03ed21-6591-491d-9489-7fa69369f2ae service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:00.534580 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1439/2839] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:42:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:42:00.798737 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9b408-3399-42a1-803a-1b20aaa330a6 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:00.800663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-76e9b408-3399-42a1-803a-1b20aaa330a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:00.816401 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9b408-3399-42a1-803a-1b20aaa330a6 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:00.821440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:00.822804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:00.823096 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:00.823734 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] PATH_INFO: `/v3/projects/cfad18f51de14ecd91e1cd94c0fdb614` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:00.825151 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfad18f51de14ecd91e1cd94c0fdb614)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:00.827182 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:00.874906 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfad18f51de14ecd91e1cd94c0fdb614', 'request_id': 'req-0519afe9-7b41-4d62-859a-2a031c6d8f67'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:00.880315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfad18f51de14ecd91e1cd94c0fdb614', 'request_id': 'req-0519afe9-7b41-4d62-859a-2a031c6d8f67'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:00.886381 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfad18f51de14ecd91e1cd94c0fdb614', 'request_id': 'req-0519afe9-7b41-4d62-859a-2a031c6d8f67'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:00.898162 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] The token cache is being invalidate because project cfad18f51de14ecd91e1cd94c0fdb614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:42:00.898446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:00.899057 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:00.900547 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1401/2840] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:42:00 2022] DELETE /identity/v3/projects/cfad18f51de14ecd91e1cd94c0fdb614 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:42:00.904606 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03ed21-6591-491d-9489-7fa69369f2ae service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:00.914900 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03ed21-6591-491d-9489-7fa69369f2ae service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:00.968625 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2871e-a02b-46ca-abec-2f0c4f265bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:00.969252 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2871e-a02b-46ca-abec-2f0c4f265bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:00.969524 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2871e-a02b-46ca-abec-2f0c4f265bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:00.969793 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2871e-a02b-46ca-abec-2f0c4f265bad admin admin] PATH_INFO: `/v3/users/3d0b8866017d49f4814b33f7a665bcc7` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:00.970891 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2871e-a02b-46ca-abec-2f0c4f265bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d0b8866017d49f4814b33f7a665bcc7)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:00.972821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2871e-a02b-46ca-abec-2f0c4f265bad admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:00.974240 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:01.018646 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e6d2871e-a02b-46ca-abec-2f0c4f265bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d0b8866017d49f4814b33f7a665bcc7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:01.025785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e6d2871e-a02b-46ca-abec-2f0c4f265bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d0b8866017d49f4814b33f7a665bcc7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:01.026416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e6d2871e-a02b-46ca-abec-2f0c4f265bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d0b8866017d49f4814b33f7a665bcc7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:01.033335 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e6d2871e-a02b-46ca-abec-2f0c4f265bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d0b8866017d49f4814b33f7a665bcc7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:01.046118 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-e6d2871e-a02b-46ca-abec-2f0c4f265bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d0b8866017d49f4814b33f7a665bcc7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:01.051887 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1440/2841] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:42:00 2022] DELETE /identity/v3/users/3d0b8866017d49f4814b33f7a665bcc7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:42:01.318187 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:01.319599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:01.335193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519afe9-7b41-4d62-859a-2a031c6d8f67 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:01.400497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:01.401112 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:01.401226 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:01.401463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] PATH_INFO: `/v3/projects/ccefb3d6dc17499ea1541f6058cc8202` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:01.402527 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccefb3d6dc17499ea1541f6058cc8202)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:01.404457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:01.406225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:01.454359 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccefb3d6dc17499ea1541f6058cc8202', 'request_id': 'req-d20dad23-0ede-4f47-8ac0-a655962a7204'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:01.460338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccefb3d6dc17499ea1541f6058cc8202', 'request_id': 'req-d20dad23-0ede-4f47-8ac0-a655962a7204'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:01.466402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccefb3d6dc17499ea1541f6058cc8202', 'request_id': 'req-d20dad23-0ede-4f47-8ac0-a655962a7204'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:01.479265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] The token cache is being invalidate because project ccefb3d6dc17499ea1541f6058cc8202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:42:01.479551 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:01.480154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-d20dad23-0ede-4f47-8ac0-a655962a7204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:01.481641 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1402/2842] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:42:01 2022] DELETE /identity/v3/projects/ccefb3d6dc17499ea1541f6058cc8202 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:42:01.486439 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff27bb4-db81-4cf5-bedc-4dc1eb02908f None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:01.486915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff27bb4-db81-4cf5-bedc-4dc1eb02908f None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:01.487025 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff27bb4-db81-4cf5-bedc-4dc1eb02908f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:01.505889 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7ff27bb4-db81-4cf5-bedc-4dc1eb02908f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:01.507585 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7ff27bb4-db81-4cf5-bedc-4dc1eb02908f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:01.564090 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1441/2843] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:01.574053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60e8a1-b3f3-4d19-92f2-95083dfef7b8 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:01.574338 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60e8a1-b3f3-4d19-92f2-95083dfef7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:01.574636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60e8a1-b3f3-4d19-92f2-95083dfef7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:01.597014 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-0f60e8a1-b3f3-4d19-92f2-95083dfef7b8 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:01.598206 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-0f60e8a1-b3f3-4d19-92f2-95083dfef7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:01.605918 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1403/2844] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:01.610112 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff27bb4-db81-4cf5-bedc-4dc1eb02908f None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:01.619712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff27bb4-db81-4cf5-bedc-4dc1eb02908f None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:01.624208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28fa32-18ba-4bd1-90a0-604e2940b882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:01.624814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28fa32-18ba-4bd1-90a0-604e2940b882 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:01.625092 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28fa32-18ba-4bd1-90a0-604e2940b882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:01.625364 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28fa32-18ba-4bd1-90a0-604e2940b882 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:01.626002 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28fa32-18ba-4bd1-90a0-604e2940b882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:42:01.626303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28fa32-18ba-4bd1-90a0-604e2940b882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:01.627877 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28fa32-18ba-4bd1-90a0-604e2940b882 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:01.639779 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1442/2845] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:42:01.643891 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60e8a1-b3f3-4d19-92f2-95083dfef7b8 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:01.653597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60e8a1-b3f3-4d19-92f2-95083dfef7b8 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:01.658123 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69d3ed-9cbd-43c5-a21a-a92c7ffb5250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:01.658738 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69d3ed-9cbd-43c5-a21a-a92c7ffb5250 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:01.659016 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69d3ed-9cbd-43c5-a21a-a92c7ffb5250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:01.659282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69d3ed-9cbd-43c5-a21a-a92c7ffb5250 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:01.659908 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69d3ed-9cbd-43c5-a21a-a92c7ffb5250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:01.662115 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69d3ed-9cbd-43c5-a21a-a92c7ffb5250 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:01.687573 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1404/2846] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:42:01 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:42:01.691787 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28fa32-18ba-4bd1-90a0-604e2940b882 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:01.702088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28fa32-18ba-4bd1-90a0-604e2940b882 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:01.758305 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7928aee-b2a8-4b34-bcd6-5aeede934055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:01.758907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7928aee-b2a8-4b34-bcd6-5aeede934055 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:01.759184 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7928aee-b2a8-4b34-bcd6-5aeede934055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:01.759454 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7928aee-b2a8-4b34-bcd6-5aeede934055 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:01.760072 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7928aee-b2a8-4b34-bcd6-5aeede934055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:01.761995 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7928aee-b2a8-4b34-bcd6-5aeede934055 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:01.773254 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:01.784761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1443/2847] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:42:01 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:42:01.789134 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69d3ed-9cbd-43c5-a21a-a92c7ffb5250 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:01.799199 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69d3ed-9cbd-43c5-a21a-a92c7ffb5250 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:01.804285 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebca38-2fd9-46e4-bd46-8ee5eb11cec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:01.804925 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebca38-2fd9-46e4-bd46-8ee5eb11cec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:01.805189 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebca38-2fd9-46e4-bd46-8ee5eb11cec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:01.805459 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebca38-2fd9-46e4-bd46-8ee5eb11cec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:01.806121 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebca38-2fd9-46e4-bd46-8ee5eb11cec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:01.807698 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebca38-2fd9-46e4-bd46-8ee5eb11cec4 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:01.817817 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1405/2848] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:42:01.822193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7928aee-b2a8-4b34-bcd6-5aeede934055 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:01.831897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7928aee-b2a8-4b34-bcd6-5aeede934055 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:01.836467 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309aa243-4399-46e5-9f3b-f00bf093ae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:01.840391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aa243-4399-46e5-9f3b-f00bf093ae5e admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:01.840391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aa243-4399-46e5-9f3b-f00bf093ae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:01.840391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aa243-4399-46e5-9f3b-f00bf093ae5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:01.840391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309aa243-4399-46e5-9f3b-f00bf093ae5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:01.840391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309aa243-4399-46e5-9f3b-f00bf093ae5e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:01.851366 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1444/2849] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:42:01.855474 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebca38-2fd9-46e4-bd46-8ee5eb11cec4 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:01.865629 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebca38-2fd9-46e4-bd46-8ee5eb11cec4 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:01.870688 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719afa7-6cf2-45f0-af98-eee83615c4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:01.871319 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719afa7-6cf2-45f0-af98-eee83615c4bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:01.871597 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719afa7-6cf2-45f0-af98-eee83615c4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:01.871866 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719afa7-6cf2-45f0-af98-eee83615c4bb admin admin] PATH_INFO: `/v3/projects/c6231abce327459ebfd6e45becb64aac/users/8619e075278e416f87a3f1f8ad4b7354/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:01.886760 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719afa7-6cf2-45f0-af98-eee83615c4bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6231abce327459ebfd6e45becb64aac, user_id=8619e075278e416f87a3f1f8ad4b7354, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:01.889425 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719afa7-6cf2-45f0-af98-eee83615c4bb admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:01.901980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1406/2850] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:42:01 2022] PUT /identity/v3/projects/c6231abce327459ebfd6e45becb64aac/users/8619e075278e416f87a3f1f8ad4b7354/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:42:01.912394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48b97c-bf60-40b7-bb3a-9ef4e5728376 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:01.912631 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48b97c-bf60-40b7-bb3a-9ef4e5728376 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:01.912905 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48b97c-bf60-40b7-bb3a-9ef4e5728376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:01.939791 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-1f48b97c-bf60-40b7-bb3a-9ef4e5728376 None None] MFA Rules not processed for user `8619e075278e416f87a3f1f8ad4b7354`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:01.941010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-1f48b97c-bf60-40b7-bb3a-9ef4e5728376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:02.003718 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1445/2851] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:02.015859 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719afa7-6cf2-45f0-af98-eee83615c4bb admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:02.017585 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4719afa7-6cf2-45f0-af98-eee83615c4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:02.073228 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719afa7-6cf2-45f0-af98-eee83615c4bb admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:02.076899 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4277e-19db-4111-814d-9ad46cb5e6f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:02.077487 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4277e-19db-4111-814d-9ad46cb5e6f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:02.077761 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4277e-19db-4111-814d-9ad46cb5e6f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:02.078029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4277e-19db-4111-814d-9ad46cb5e6f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:02.086126 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4277e-19db-4111-814d-9ad46cb5e6f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:02.087532 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4277e-19db-4111-814d-9ad46cb5e6f0 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:02.096212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1407/2852] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:42:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:42:02.114728 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48b97c-bf60-40b7-bb3a-9ef4e5728376 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:02.177159 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48b97c-bf60-40b7-bb3a-9ef4e5728376 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:02.182338 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:02.182948 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:02.183222 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:02.183487 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] PATH_INFO: `/v3/users/8619e075278e416f87a3f1f8ad4b7354` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:02.184616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8619e075278e416f87a3f1f8ad4b7354)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:02.186495 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:02.233616 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8619e075278e416f87a3f1f8ad4b7354'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:02.240883 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8619e075278e416f87a3f1f8ad4b7354'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:02.242186 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8619e075278e416f87a3f1f8ad4b7354'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:02.248923 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8619e075278e416f87a3f1f8ad4b7354'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:02.259303 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:02.261967 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8619e075278e416f87a3f1f8ad4b7354'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:02.269162 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1446/2853] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:42:02 2022] DELETE /identity/v3/users/8619e075278e416f87a3f1f8ad4b7354 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:42:02.273534 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0a2fc-76e8-4197-9136-bc246f934d24 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:02.273820 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0a2fc-76e8-4197-9136-bc246f934d24 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:02.274108 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0a2fc-76e8-4197-9136-bc246f934d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:02.293177 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e9f0a2fc-76e8-4197-9136-bc246f934d24 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:02.294374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e9f0a2fc-76e8-4197-9136-bc246f934d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:02.317531 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:02.351785 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1408/2854] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:02.361474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:02.362890 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:02.421515 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e55bda-3b31-4a6e-bd7d-c52de835ddcd admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:02.425153 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542803d-61b3-495c-8a57-d663cce41663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:02.425750 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542803d-61b3-495c-8a57-d663cce41663 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:02.426021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542803d-61b3-495c-8a57-d663cce41663 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:02.426284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542803d-61b3-495c-8a57-d663cce41663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:02.435455 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542803d-61b3-495c-8a57-d663cce41663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:02.436907 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542803d-61b3-495c-8a57-d663cce41663 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:02.446021 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1447/2855] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:42:02.777608 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0a2fc-76e8-4197-9136-bc246f934d24 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:02.779053 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e9f0a2fc-76e8-4197-9136-bc246f934d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:02.793214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0a2fc-76e8-4197-9136-bc246f934d24 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:02.797986 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:02.798619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:02.799378 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:02.799663 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] PATH_INFO: `/v3/projects/c6231abce327459ebfd6e45becb64aac` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:02.800954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6231abce327459ebfd6e45becb64aac)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:02.802868 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:02.848515 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6231abce327459ebfd6e45becb64aac', 'request_id': 'req-0df83a02-c686-4bcc-8283-d4169e510f74'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:02.854545 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6231abce327459ebfd6e45becb64aac', 'request_id': 'req-0df83a02-c686-4bcc-8283-d4169e510f74'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:02.860588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6231abce327459ebfd6e45becb64aac', 'request_id': 'req-0df83a02-c686-4bcc-8283-d4169e510f74'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:02.873224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] The token cache is being invalidate because project c6231abce327459ebfd6e45becb64aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:42:02.873507 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:02.874120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-0df83a02-c686-4bcc-8283-d4169e510f74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:02.875599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1409/2856] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:42:02 2022] DELETE /identity/v3/projects/c6231abce327459ebfd6e45becb64aac => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:42:02.880511 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906af375-46b0-4666-83a2-389973cf6c97 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:02.880993 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906af375-46b0-4666-83a2-389973cf6c97 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:02.881101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906af375-46b0-4666-83a2-389973cf6c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:02.900358 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-906af375-46b0-4666-83a2-389973cf6c97 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:02.901549 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-906af375-46b0-4666-83a2-389973cf6c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:02.959269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1448/2857] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:02.967384 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a877163-c47d-4ffa-88a3-456b80f595cd None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:02.967666 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a877163-c47d-4ffa-88a3-456b80f595cd None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:02.967936 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a877163-c47d-4ffa-88a3-456b80f595cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:02.986952 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4a877163-c47d-4ffa-88a3-456b80f595cd None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:02.988111 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4a877163-c47d-4ffa-88a3-456b80f595cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:02.995647 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1410/2858] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:02.999628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906af375-46b0-4666-83a2-389973cf6c97 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.009541 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906af375-46b0-4666-83a2-389973cf6c97 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.014082 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963c6eb-c8cb-4131-ba1c-ab0bcda0d05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.014673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963c6eb-c8cb-4131-ba1c-ab0bcda0d05c admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.014957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963c6eb-c8cb-4131-ba1c-ab0bcda0d05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.015228 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963c6eb-c8cb-4131-ba1c-ab0bcda0d05c admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.015867 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963c6eb-c8cb-4131-ba1c-ab0bcda0d05c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:42:03.016170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963c6eb-c8cb-4131-ba1c-ab0bcda0d05c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.017784 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963c6eb-c8cb-4131-ba1c-ab0bcda0d05c admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.029605 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1449/2859] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:42:03.033580 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a877163-c47d-4ffa-88a3-456b80f595cd None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.042805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a877163-c47d-4ffa-88a3-456b80f595cd None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.047294 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a3c60-fd68-430e-8aae-6cf712f282ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.047870 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a3c60-fd68-430e-8aae-6cf712f282ec admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.048145 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a3c60-fd68-430e-8aae-6cf712f282ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.048440 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a3c60-fd68-430e-8aae-6cf712f282ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.049066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a3c60-fd68-430e-8aae-6cf712f282ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.050961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a3c60-fd68-430e-8aae-6cf712f282ec admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.077723 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1411/2860] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:42:03 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:42:03.081729 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963c6eb-c8cb-4131-ba1c-ab0bcda0d05c admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.092198 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963c6eb-c8cb-4131-ba1c-ab0bcda0d05c admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.146125 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cba4fc-f715-4c4e-8584-2db67ec70736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.146792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cba4fc-f715-4c4e-8584-2db67ec70736 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.147170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cba4fc-f715-4c4e-8584-2db67ec70736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.148099 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cba4fc-f715-4c4e-8584-2db67ec70736 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.148785 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cba4fc-f715-4c4e-8584-2db67ec70736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.150738 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cba4fc-f715-4c4e-8584-2db67ec70736 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.173911 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1450/2861] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:42:03 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:42:03.175345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a3c60-fd68-430e-8aae-6cf712f282ec admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.185651 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a3c60-fd68-430e-8aae-6cf712f282ec admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.190261 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5856df-b995-4618-8afa-5291da12171d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.190856 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5856df-b995-4618-8afa-5291da12171d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.191139 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5856df-b995-4618-8afa-5291da12171d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.191414 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5856df-b995-4618-8afa-5291da12171d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.192083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5856df-b995-4618-8afa-5291da12171d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.193722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5856df-b995-4618-8afa-5291da12171d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.205214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1412/2862] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:42:03.209161 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cba4fc-f715-4c4e-8584-2db67ec70736 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.219999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cba4fc-f715-4c4e-8584-2db67ec70736 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.224566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b56bf-a0fe-42bf-b13b-4e3cec699a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.225540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b56bf-a0fe-42bf-b13b-4e3cec699a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.225540 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b56bf-a0fe-42bf-b13b-4e3cec699a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.225736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b56bf-a0fe-42bf-b13b-4e3cec699a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.226400 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b56bf-a0fe-42bf-b13b-4e3cec699a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.228088 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b56bf-a0fe-42bf-b13b-4e3cec699a70 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.240406 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1451/2863] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:42:03.243102 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5856df-b995-4618-8afa-5291da12171d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.253659 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5856df-b995-4618-8afa-5291da12171d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.258827 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe493b-5695-4130-b36e-319f846134ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.259470 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe493b-5695-4130-b36e-319f846134ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.259752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe493b-5695-4130-b36e-319f846134ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.260026 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe493b-5695-4130-b36e-319f846134ec admin admin] PATH_INFO: `/v3/projects/12a612f3d4f64eaab10442cec5c6646e/users/b72d0b5ada224cc8bd6fad05713b42af/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.274872 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe493b-5695-4130-b36e-319f846134ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a612f3d4f64eaab10442cec5c6646e, user_id=b72d0b5ada224cc8bd6fad05713b42af, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.277550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe493b-5695-4130-b36e-319f846134ec admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.289974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1413/2864] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:42:03 2022] PUT /identity/v3/projects/12a612f3d4f64eaab10442cec5c6646e/users/b72d0b5ada224cc8bd6fad05713b42af/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:42:03.354405 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d33e9-e22d-44ac-9f93-9a13300ef2cb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.354684 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d33e9-e22d-44ac-9f93-9a13300ef2cb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.354951 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d33e9-e22d-44ac-9f93-9a13300ef2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.362079 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:03.386859 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-069d33e9-e22d-44ac-9f93-9a13300ef2cb None None] MFA Rules not processed for user `b72d0b5ada224cc8bd6fad05713b42af`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:03.388086 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-069d33e9-e22d-44ac-9f93-9a13300ef2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:03.450468 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1452/2865] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:03.454890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe493b-5695-4130-b36e-319f846134ec admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.502041 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:03.517720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe493b-5695-4130-b36e-319f846134ec admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.522520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362afbe-df5f-47b9-acca-c4fc86f32b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.523119 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362afbe-df5f-47b9-acca-c4fc86f32b11 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.523400 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362afbe-df5f-47b9-acca-c4fc86f32b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.523671 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362afbe-df5f-47b9-acca-c4fc86f32b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.524299 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362afbe-df5f-47b9-acca-c4fc86f32b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.526298 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362afbe-df5f-47b9-acca-c4fc86f32b11 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.552457 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1414/2866] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:42:03 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:42:03.557068 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d33e9-e22d-44ac-9f93-9a13300ef2cb None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.568279 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d33e9-e22d-44ac-9f93-9a13300ef2cb None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.623030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef65272-6194-43d9-849b-233e51970890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.623627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef65272-6194-43d9-849b-233e51970890 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.623902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef65272-6194-43d9-849b-233e51970890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.624170 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef65272-6194-43d9-849b-233e51970890 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.624836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef65272-6194-43d9-849b-233e51970890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.626714 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef65272-6194-43d9-849b-233e51970890 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.647639 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1453/2867] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:42:03 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:42:03.651612 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362afbe-df5f-47b9-acca-c4fc86f32b11 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.662225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362afbe-df5f-47b9-acca-c4fc86f32b11 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.667295 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78375623-7729-49bb-969f-81ba0950a51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.667883 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78375623-7729-49bb-969f-81ba0950a51d admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.668154 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78375623-7729-49bb-969f-81ba0950a51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.668453 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78375623-7729-49bb-969f-81ba0950a51d admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.669085 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78375623-7729-49bb-969f-81ba0950a51d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.670668 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78375623-7729-49bb-969f-81ba0950a51d admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.681677 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1415/2868] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:42:03.685600 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef65272-6194-43d9-849b-233e51970890 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.695655 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef65272-6194-43d9-849b-233e51970890 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.700293 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a06ac1-2f49-4c53-a260-5524930d596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.700969 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a06ac1-2f49-4c53-a260-5524930d596e admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.701247 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a06ac1-2f49-4c53-a260-5524930d596e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.701519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a06ac1-2f49-4c53-a260-5524930d596e admin admin] PATH_INFO: `/v3/projects/4bb6d7659ba34bfca6ee0495ce1f1f0a/users/4336aa9152e74a89ba249015171cd413/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.716174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a06ac1-2f49-4c53-a260-5524930d596e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6d7659ba34bfca6ee0495ce1f1f0a, user_id=4336aa9152e74a89ba249015171cd413, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.718864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a06ac1-2f49-4c53-a260-5524930d596e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.732824 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1454/2869] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:42:03 2022] PUT /identity/v3/projects/4bb6d7659ba34bfca6ee0495ce1f1f0a/users/4336aa9152e74a89ba249015171cd413/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:42:03.736919 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78375623-7729-49bb-969f-81ba0950a51d admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.748699 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78375623-7729-49bb-969f-81ba0950a51d admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.806290 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9e786-1f23-496f-8a07-2d6c1de20b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.806968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9e786-1f23-496f-8a07-2d6c1de20b40 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.806968 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9e786-1f23-496f-8a07-2d6c1de20b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.807166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9e786-1f23-496f-8a07-2d6c1de20b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.807656 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9e786-1f23-496f-8a07-2d6c1de20b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.809303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9e786-1f23-496f-8a07-2d6c1de20b40 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.819615 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1416/2870] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:42:03.823892 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a06ac1-2f49-4c53-a260-5524930d596e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.834044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a06ac1-2f49-4c53-a260-5524930d596e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:03.839357 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a59bb-399a-4320-ae94-8c2252b15fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:03.840010 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a59bb-399a-4320-ae94-8c2252b15fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.840289 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a59bb-399a-4320-ae94-8c2252b15fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.840590 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a59bb-399a-4320-ae94-8c2252b15fc0 admin admin] PATH_INFO: `/v3/projects/4bb6d7659ba34bfca6ee0495ce1f1f0a/users/4336aa9152e74a89ba249015171cd413/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.842543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a59bb-399a-4320-ae94-8c2252b15fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb6d7659ba34bfca6ee0495ce1f1f0a, user_id=4336aa9152e74a89ba249015171cd413, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:03.845269 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a59bb-399a-4320-ae94-8c2252b15fc0 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:03.855712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1455/2871] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:42:03 2022] PUT /identity/v3/projects/4bb6d7659ba34bfca6ee0495ce1f1f0a/users/4336aa9152e74a89ba249015171cd413/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:42:03.865665 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4574df4-d356-4867-a15b-122bb748997b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:03.865980 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4574df4-d356-4867-a15b-122bb748997b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:03.866251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4574df4-d356-4867-a15b-122bb748997b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:03.893422 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-a4574df4-d356-4867-a15b-122bb748997b None None] MFA Rules not processed for user `4336aa9152e74a89ba249015171cd413`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:03.894601 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a4574df4-d356-4867-a15b-122bb748997b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:03.960376 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1417/2872] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:03.970827 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a59bb-399a-4320-ae94-8c2252b15fc0 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:03.972658 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-4c5a59bb-399a-4320-ae94-8c2252b15fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:04.018988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:04.030726 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a59bb-399a-4320-ae94-8c2252b15fc0 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:04.034336 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca2f4d-cb33-47e2-aea6-fea4989487b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:04.034925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2f4d-cb33-47e2-aea6-fea4989487b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:04.035203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2f4d-cb33-47e2-aea6-fea4989487b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:04.035509 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2f4d-cb33-47e2-aea6-fea4989487b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:04.089838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca2f4d-cb33-47e2-aea6-fea4989487b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:04.091203 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca2f4d-cb33-47e2-aea6-fea4989487b9 service cinder] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:04.099850 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1456/2873] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:42:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:42:07.861653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4574df4-d356-4867-a15b-122bb748997b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:07.906042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:07.924402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4574df4-d356-4867-a15b-122bb748997b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:07.928954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:07.929584 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:07.929886 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:07.930161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] PATH_INFO: `/v3/users/b72d0b5ada224cc8bd6fad05713b42af` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:07.931268 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b72d0b5ada224cc8bd6fad05713b42af)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:07.933233 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:07.980900 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b72d0b5ada224cc8bd6fad05713b42af'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:07.987644 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b72d0b5ada224cc8bd6fad05713b42af'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:07.988297 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b72d0b5ada224cc8bd6fad05713b42af'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:07.995345 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b72d0b5ada224cc8bd6fad05713b42af'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.007040 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b72d0b5ada224cc8bd6fad05713b42af'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.012719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1418/2874] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:42:07 2022] DELETE /identity/v3/users/b72d0b5ada224cc8bd6fad05713b42af => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:42:08.017164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c24a7-be60-404b-a681-c820dc5225aa None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:08.017627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c24a7-be60-404b-a681-c820dc5225aa None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:08.017743 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c24a7-be60-404b-a681-c820dc5225aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:08.038753 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-988c24a7-be60-404b-a681-c820dc5225aa None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:08.039943 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-988c24a7-be60-404b-a681-c820dc5225aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:08.096543 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1457/2875] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:08.106722 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:08.108166 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:08.165171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad6e89-ce7c-4884-b4e7-77167465fd6f admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:08.168815 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f295b80-4a9b-4c55-8923-98e6a5b02d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:08.169410 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f295b80-4a9b-4c55-8923-98e6a5b02d73 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:08.169693 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f295b80-4a9b-4c55-8923-98e6a5b02d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:08.169966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f295b80-4a9b-4c55-8923-98e6a5b02d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:08.178807 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f295b80-4a9b-4c55-8923-98e6a5b02d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:08.180264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f295b80-4a9b-4c55-8923-98e6a5b02d73 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:08.189492 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1419/2876] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:42:08.490349 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c24a7-be60-404b-a681-c820dc5225aa None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:08.492333 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-988c24a7-be60-404b-a681-c820dc5225aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:08.504049 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c24a7-be60-404b-a681-c820dc5225aa None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:08.509691 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:08.510402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:08.510626 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:08.510896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] PATH_INFO: `/v3/projects/12a612f3d4f64eaab10442cec5c6646e` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:08.512061 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a612f3d4f64eaab10442cec5c6646e)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:08.513991 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:08.573368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a612f3d4f64eaab10442cec5c6646e', 'request_id': 'req-31195571-14a4-454d-98b4-9ce1012ef57e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.585711 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a612f3d4f64eaab10442cec5c6646e', 'request_id': 'req-31195571-14a4-454d-98b4-9ce1012ef57e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.593777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a612f3d4f64eaab10442cec5c6646e', 'request_id': 'req-31195571-14a4-454d-98b4-9ce1012ef57e'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.607701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] The token cache is being invalidate because project 12a612f3d4f64eaab10442cec5c6646e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:42:08.608148 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.608896 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.610550 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1458/2877] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:42:08 2022] DELETE /identity/v3/projects/12a612f3d4f64eaab10442cec5c6646e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:42:08.614836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f295b80-4a9b-4c55-8923-98e6a5b02d73 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:08.625271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f295b80-4a9b-4c55-8923-98e6a5b02d73 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:08.682091 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877133b-42d2-4d57-8b5a-5971b77b16f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:08.682709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877133b-42d2-4d57-8b5a-5971b77b16f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:08.682984 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877133b-42d2-4d57-8b5a-5971b77b16f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:08.683252 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877133b-42d2-4d57-8b5a-5971b77b16f3 admin admin] PATH_INFO: `/v3/users/4336aa9152e74a89ba249015171cd413` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:08.684346 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877133b-42d2-4d57-8b5a-5971b77b16f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4336aa9152e74a89ba249015171cd413)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:08.686275 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877133b-42d2-4d57-8b5a-5971b77b16f3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:08.692239 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:08.731058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9877133b-42d2-4d57-8b5a-5971b77b16f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4336aa9152e74a89ba249015171cd413'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.738083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9877133b-42d2-4d57-8b5a-5971b77b16f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4336aa9152e74a89ba249015171cd413'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.738743 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9877133b-42d2-4d57-8b5a-5971b77b16f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4336aa9152e74a89ba249015171cd413'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.746469 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9877133b-42d2-4d57-8b5a-5971b77b16f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4336aa9152e74a89ba249015171cd413'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.759225 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-9877133b-42d2-4d57-8b5a-5971b77b16f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4336aa9152e74a89ba249015171cd413'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:08.765066 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1420/2878] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:42:08 2022] DELETE /identity/v3/users/4336aa9152e74a89ba249015171cd413 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 19:42:09.046912 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:09.048685 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:09.068518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31195571-14a4-454d-98b4-9ce1012ef57e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:09.086659 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:09.126935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:09.127574 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:09.127852 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:09.128122 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] PATH_INFO: `/v3/projects/4bb6d7659ba34bfca6ee0495ce1f1f0a` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:09.129288 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bb6d7659ba34bfca6ee0495ce1f1f0a)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:09.131195 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:09.182840 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bb6d7659ba34bfca6ee0495ce1f1f0a', 'request_id': 'req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:09.189266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bb6d7659ba34bfca6ee0495ce1f1f0a', 'request_id': 'req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:09.196768 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bb6d7659ba34bfca6ee0495ce1f1f0a', 'request_id': 'req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:09.211757 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] The token cache is being invalidate because project 4bb6d7659ba34bfca6ee0495ce1f1f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81402) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:42:09.212044 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:09.212706 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1ad1832a-6952-4f88-bffa-75b2ae1cc0e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:09.214185 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1459/2879] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:42:09 2022] DELETE /identity/v3/projects/4bb6d7659ba34bfca6ee0495ce1f1f0a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:42:09.220158 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d4dc-37f7-4faa-9e34-e5f5e1e1d25f None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:09.221114 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d4dc-37f7-4faa-9e34-e5f5e1e1d25f None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:09.221444 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d4dc-37f7-4faa-9e34-e5f5e1e1d25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:09.245069 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-7fb6d4dc-37f7-4faa-9e34-e5f5e1e1d25f None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:09.246949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7fb6d4dc-37f7-4faa-9e34-e5f5e1e1d25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:09.310465 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1421/2880] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:09.318075 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322003f-c3eb-4095-9562-4ed4aefcf180 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:09.318381 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322003f-c3eb-4095-9562-4ed4aefcf180 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:09.318808 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322003f-c3eb-4095-9562-4ed4aefcf180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:09.338996 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-7322003f-c3eb-4095-9562-4ed4aefcf180 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:09.340176 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-7322003f-c3eb-4095-9562-4ed4aefcf180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:09.348023 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1460/2881] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:09.353438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d4dc-37f7-4faa-9e34-e5f5e1e1d25f None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:09.363317 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d4dc-37f7-4faa-9e34-e5f5e1e1d25f None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:09.368326 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44701b-bbde-4127-a4f4-f7717c77fd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:09.369093 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44701b-bbde-4127-a4f4-f7717c77fd0c admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:09.369568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44701b-bbde-4127-a4f4-f7717c77fd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:09.369889 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44701b-bbde-4127-a4f4-f7717c77fd0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:09.370763 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44701b-bbde-4127-a4f4-f7717c77fd0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81403) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:42:09.371256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44701b-bbde-4127-a4f4-f7717c77fd0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:09.373781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44701b-bbde-4127-a4f4-f7717c77fd0c admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:09.386265 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1422/2882] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:42:09.390645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322003f-c3eb-4095-9562-4ed4aefcf180 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:09.400508 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322003f-c3eb-4095-9562-4ed4aefcf180 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:09.405120 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b107f-e381-4982-9c04-5fed1169c613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:09.405719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b107f-e381-4982-9c04-5fed1169c613 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:09.405995 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b107f-e381-4982-9c04-5fed1169c613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:09.406263 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b107f-e381-4982-9c04-5fed1169c613 admin admin] PATH_INFO: `/v3/projects` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:09.406897 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b107f-e381-4982-9c04-5fed1169c613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:09.408814 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b107f-e381-4982-9c04-5fed1169c613 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:09.435697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1461/2883] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:42:09 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:42:09.439778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44701b-bbde-4127-a4f4-f7717c77fd0c admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:09.449550 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44701b-bbde-4127-a4f4-f7717c77fd0c admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:09.504438 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923f843-3bc1-41c4-b945-e51d13c1dcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:09.505034 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f843-3bc1-41c4-b945-e51d13c1dcb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:09.505313 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f843-3bc1-41c4-b945-e51d13c1dcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:09.505581 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f843-3bc1-41c4-b945-e51d13c1dcb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:09.506213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923f843-3bc1-41c4-b945-e51d13c1dcb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:09.508161 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923f843-3bc1-41c4-b945-e51d13c1dcb2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:09.529212 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1423/2884] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:42:09 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:42:09.533271 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b107f-e381-4982-9c04-5fed1169c613 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:09.544632 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b107f-e381-4982-9c04-5fed1169c613 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:09.549200 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c02701-9ab8-4e51-b521-912f071f9c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:09.549821 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c02701-9ab8-4e51-b521-912f071f9c57 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:09.550095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c02701-9ab8-4e51-b521-912f071f9c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:09.550363 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c02701-9ab8-4e51-b521-912f071f9c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:09.551000 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c02701-9ab8-4e51-b521-912f071f9c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:09.552611 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c02701-9ab8-4e51-b521-912f071f9c57 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:09.562309 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1462/2885] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:42:09.566387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923f843-3bc1-41c4-b945-e51d13c1dcb2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:09.571593 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:09.577989 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923f843-3bc1-41c4-b945-e51d13c1dcb2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:09.584042 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d980079-0a7a-4125-b7e7-67263a23a021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:09.584672 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d980079-0a7a-4125-b7e7-67263a23a021 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:09.584949 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d980079-0a7a-4125-b7e7-67263a23a021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:09.585216 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d980079-0a7a-4125-b7e7-67263a23a021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:09.586377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d980079-0a7a-4125-b7e7-67263a23a021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:09.587966 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d980079-0a7a-4125-b7e7-67263a23a021 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:09.598033 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1424/2886] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:42:09.602608 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c02701-9ab8-4e51-b521-912f071f9c57 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:09.612621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c02701-9ab8-4e51-b521-912f071f9c57 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:09.617459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243657f7-3a09-4906-9209-98fa79c8baa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:09.618107 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243657f7-3a09-4906-9209-98fa79c8baa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:09.618384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243657f7-3a09-4906-9209-98fa79c8baa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:09.618653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243657f7-3a09-4906-9209-98fa79c8baa7 admin admin] PATH_INFO: `/v3/projects/fb97895932894614bcc160f59a8c99ee/users/fe6ef93587e545758cf252145c65dde7/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:09.633749 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243657f7-3a09-4906-9209-98fa79c8baa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb97895932894614bcc160f59a8c99ee, user_id=fe6ef93587e545758cf252145c65dde7, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:09.636361 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243657f7-3a09-4906-9209-98fa79c8baa7 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:09.651671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1463/2887] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:42:09 2022] PUT /identity/v3/projects/fb97895932894614bcc160f59a8c99ee/users/fe6ef93587e545758cf252145c65dde7/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:42:09.663768 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc275bb1-569e-40c7-8bee-fb200a754524 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:09.664258 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc275bb1-569e-40c7-8bee-fb200a754524 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:09.664423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc275bb1-569e-40c7-8bee-fb200a754524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:09.698462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-dc275bb1-569e-40c7-8bee-fb200a754524 None None] MFA Rules not processed for user `fe6ef93587e545758cf252145c65dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:09.699752 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dc275bb1-569e-40c7-8bee-fb200a754524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:09.781802 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1425/2888] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:09.793715 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243657f7-3a09-4906-9209-98fa79c8baa7 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:09.795391 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-243657f7-3a09-4906-9209-98fa79c8baa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:09.852683 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:09.879627 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243657f7-3a09-4906-9209-98fa79c8baa7 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:09.883848 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d0449-e8c4-407e-b687-c340612aa9a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a9b371434243daa4176215daefb79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:09.884483 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d0449-e8c4-407e-b687-c340612aa9a6 service glance] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:09.884760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d0449-e8c4-407e-b687-c340612aa9a6 service glance] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:09.885032 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d0449-e8c4-407e-b687-c340612aa9a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:09.893563 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d0449-e8c4-407e-b687-c340612aa9a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:09.894988 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d0449-e8c4-407e-b687-c340612aa9a6 service glance] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:09.905039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1464/2889] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:42:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:42:13.971950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc275bb1-569e-40c7-8bee-fb200a754524 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:13.973446 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-dc275bb1-569e-40c7-8bee-fb200a754524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:14.050087 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc275bb1-569e-40c7-8bee-fb200a754524 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:14.056193 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3ad61-db40-40b8-b3d2-af17f4d96d1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:14.056950 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3ad61-db40-40b8-b3d2-af17f4d96d1d service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:14.057281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3ad61-db40-40b8-b3d2-af17f4d96d1d service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:14.057604 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3ad61-db40-40b8-b3d2-af17f4d96d1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:14.059494 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-35c3ad61-db40-40b8-b3d2-af17f4d96d1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:14.072920 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3ad61-db40-40b8-b3d2-af17f4d96d1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:14.074504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3ad61-db40-40b8-b3d2-af17f4d96d1d service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:14.085135 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1426/2890] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:42:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:42:19.901125 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=13 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:24.080504 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=70 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:31.921154 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25a06-a9c5-47ab-a937-47d8f9f72774 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:31.921672 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25a06-a9c5-47ab-a937-47d8f9f72774 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:31.921778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25a06-a9c5-47ab-a937-47d8f9f72774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:31.941174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-01a25a06-a9c5-47ab-a937-47d8f9f72774 None None] MFA Rules not processed for user `c5b46acc6b57473fa4cab2e37b761748`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:31.942368 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-01a25a06-a9c5-47ab-a937-47d8f9f72774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:31.996389 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1465/2891] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:32.225669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3ad61-db40-40b8-b3d2-af17f4d96d1d service nova] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:32.227054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-35c3ad61-db40-40b8-b3d2-af17f4d96d1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:32.284638 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3ad61-db40-40b8-b3d2-af17f4d96d1d service nova] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:32.288388 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8415f-d405-4c10-ad84-755e50854ddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:32.288979 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8415f-d405-4c10-ad84-755e50854ddb service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:32.289252 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8415f-d405-4c10-ad84-755e50854ddb service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:32.289520 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8415f-d405-4c10-ad84-755e50854ddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:32.291209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-61a8415f-d405-4c10-ad84-755e50854ddb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:32.301082 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8415f-d405-4c10-ad84-755e50854ddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:32.302488 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8415f-d405-4c10-ad84-755e50854ddb service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:32.310279 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1427/2892] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:42:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:42:41.989255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=25 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:42.306795 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=26 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:42:50.303335 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a25a06-a9c5-47ab-a937-47d8f9f72774 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:50.363770 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a25a06-a9c5-47ab-a937-47d8f9f72774 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:50.368384 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:50.369015 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:50.369290 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:50.369558 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] PATH_INFO: `/v3/users/fe6ef93587e545758cf252145c65dde7` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:50.370660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe6ef93587e545758cf252145c65dde7)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:50.372647 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:50.425759 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe6ef93587e545758cf252145c65dde7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:50.432456 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe6ef93587e545758cf252145c65dde7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:50.433113 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe6ef93587e545758cf252145c65dde7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:50.441645 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe6ef93587e545758cf252145c65dde7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:50.454220 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe6ef93587e545758cf252145c65dde7'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:50.459989 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1466/2893] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:42:50 2022] DELETE /identity/v3/users/fe6ef93587e545758cf252145c65dde7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:42:50.465122 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b1838-d297-441e-aad6-fbe19278b3a4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:50.465625 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b1838-d297-441e-aad6-fbe19278b3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:50.465733 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b1838-d297-441e-aad6-fbe19278b3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:50.484137 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-d79b1838-d297-441e-aad6-fbe19278b3a4 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:42:50.485331 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d79b1838-d297-441e-aad6-fbe19278b3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:50.541827 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1428/2894] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:42:50.551243 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:50.552665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:50.608512 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fbf03-27f2-4a9d-a12e-37ee8f8804e9 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:50.612026 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74467865-82aa-465d-8c95-7a826b057e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:50.612642 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467865-82aa-465d-8c95-7a826b057e2a service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:50.612920 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467865-82aa-465d-8c95-7a826b057e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:50.613189 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467865-82aa-465d-8c95-7a826b057e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:50.622466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74467865-82aa-465d-8c95-7a826b057e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:50.623876 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74467865-82aa-465d-8c95-7a826b057e2a service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:50.632999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1467/2895] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 19:42:50.898236 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b1838-d297-441e-aad6-fbe19278b3a4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:42:50.899649 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-d79b1838-d297-441e-aad6-fbe19278b3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:42:50.914209 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b1838-d297-441e-aad6-fbe19278b3a4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:42:50.919433 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:42:50.920054 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:42:50.920328 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:42:50.920622 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] PATH_INFO: `/v3/projects/fb97895932894614bcc160f59a8c99ee` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:42:50.929369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb97895932894614bcc160f59a8c99ee)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:42:50.931377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:42:50.976611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb97895932894614bcc160f59a8c99ee', 'request_id': 'req-6d1c2e0c-8265-417e-bbce-57206e0545d6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:50.982626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb97895932894614bcc160f59a8c99ee', 'request_id': 'req-6d1c2e0c-8265-417e-bbce-57206e0545d6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:50.988703 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb97895932894614bcc160f59a8c99ee', 'request_id': 'req-6d1c2e0c-8265-417e-bbce-57206e0545d6'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:51.001147 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] The token cache is being invalidate because project fb97895932894614bcc160f59a8c99ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:42:51.001442 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:51.002058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:42:51.003508 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1429/2896] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:42:50 2022] DELETE /identity/v3/projects/fb97895932894614bcc160f59a8c99ee => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:43:00.628902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:43:00.998251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:43:16.011149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74467865-82aa-465d-8c95-7a826b057e2a service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:16.012567 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-74467865-82aa-465d-8c95-7a826b057e2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:16.080449 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74467865-82aa-465d-8c95-7a826b057e2a service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:16.137934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d344f5-2263-4dcb-9161-b81fd1210180 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:16.138534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d344f5-2263-4dcb-9161-b81fd1210180 service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:16.138812 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d344f5-2263-4dcb-9161-b81fd1210180 service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:16.139081 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d344f5-2263-4dcb-9161-b81fd1210180 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:16.140856 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a4d344f5-2263-4dcb-9161-b81fd1210180 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:16.151630 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d344f5-2263-4dcb-9161-b81fd1210180 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:16.153217 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d344f5-2263-4dcb-9161-b81fd1210180 service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:16.163087 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1468/2897] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:43:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:43:26.158726 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=58 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:43:43.683423 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:43.684842 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:43.701940 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c2e0c-8265-417e-bbce-57206e0545d6 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:43.706869 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9d23c-e887-46b1-bfaf-f6ef9fbfa58e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:43.707506 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9d23c-e887-46b1-bfaf-f6ef9fbfa58e service placement] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:43.707787 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9d23c-e887-46b1-bfaf-f6ef9fbfa58e service placement] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:43.708088 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9d23c-e887-46b1-bfaf-f6ef9fbfa58e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:43.709833 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-5db9d23c-e887-46b1-bfaf-f6ef9fbfa58e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:43.721481 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9d23c-e887-46b1-bfaf-f6ef9fbfa58e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:43.723361 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9d23c-e887-46b1-bfaf-f6ef9fbfa58e service placement] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:43.738586 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1430/2898] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:43:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:43:43.741942 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d344f5-2263-4dcb-9161-b81fd1210180 service placement] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:43.743477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a4d344f5-2263-4dcb-9161-b81fd1210180 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:43.808568 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d344f5-2263-4dcb-9161-b81fd1210180 service placement] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:43.813782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f718bc1-0d09-4762-a825-7351ad172a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:43.814408 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f718bc1-0d09-4762-a825-7351ad172a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:43.814689 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f718bc1-0d09-4762-a825-7351ad172a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:43.814959 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f718bc1-0d09-4762-a825-7351ad172a29 admin admin] PATH_INFO: `/v3/users/c5b46acc6b57473fa4cab2e37b761748` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:43.816064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f718bc1-0d09-4762-a825-7351ad172a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b46acc6b57473fa4cab2e37b761748)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:43.818009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f718bc1-0d09-4762-a825-7351ad172a29 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:43.862589 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1f718bc1-0d09-4762-a825-7351ad172a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b46acc6b57473fa4cab2e37b761748'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:43:43.869388 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1f718bc1-0d09-4762-a825-7351ad172a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b46acc6b57473fa4cab2e37b761748'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:43:43.870039 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1f718bc1-0d09-4762-a825-7351ad172a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b46acc6b57473fa4cab2e37b761748'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:43:43.877183 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1f718bc1-0d09-4762-a825-7351ad172a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b46acc6b57473fa4cab2e37b761748'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:43:43.890719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-1f718bc1-0d09-4762-a825-7351ad172a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b46acc6b57473fa4cab2e37b761748'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:43:43.896415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1469/2899] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:43:43 2022] DELETE /identity/v3/users/c5b46acc6b57473fa4cab2e37b761748 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:43:44.064653 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9d23c-e887-46b1-bfaf-f6ef9fbfa58e service placement] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:44.077131 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9d23c-e887-46b1-bfaf-f6ef9fbfa58e service placement] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:44.133874 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:44.134503 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:44.134778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:44.135047 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] PATH_INFO: `/v3/projects/9fb45af355274456bc3759ebb76f6b72` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:44.136146 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb45af355274456bc3759ebb76f6b72)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:44.138143 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:44.183022 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb45af355274456bc3759ebb76f6b72', 'request_id': 'req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:43:44.188452 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb45af355274456bc3759ebb76f6b72', 'request_id': 'req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:43:44.194814 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb45af355274456bc3759ebb76f6b72', 'request_id': 'req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:43:44.207001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] The token cache is being invalidate because project 9fb45af355274456bc3759ebb76f6b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:43:44.207362 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:43:44.207998 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-cb3222a6-25fc-4ab2-bc69-0e4e4182024b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:43:44.209502 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1431/2900] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:43:44 2022] DELETE /identity/v3/projects/9fb45af355274456bc3759ebb76f6b72 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:43:44.215477 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9619f1-fdd7-499e-b51a-ad3169e66b77 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:44.215761 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9619f1-fdd7-499e-b51a-ad3169e66b77 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:44.216030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9619f1-fdd7-499e-b51a-ad3169e66b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:44.236121 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-2a9619f1-fdd7-499e-b51a-ad3169e66b77 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:43:44.237324 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-2a9619f1-fdd7-499e-b51a-ad3169e66b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:44.294352 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1470/2901] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:43:44.302846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca889dd3-1280-4533-8209-22ab273a70c4 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:44.303130 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca889dd3-1280-4533-8209-22ab273a70c4 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:44.303402 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca889dd3-1280-4533-8209-22ab273a70c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:44.322171 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-ca889dd3-1280-4533-8209-22ab273a70c4 None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:43:44.323357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-ca889dd3-1280-4533-8209-22ab273a70c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:44.331562 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1432/2902] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:43:44.335633 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9619f1-fdd7-499e-b51a-ad3169e66b77 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:44.345621 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9619f1-fdd7-499e-b51a-ad3169e66b77 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:44.350169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1d1ce-c1ea-43cd-ac46-8d03b37ef571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:44.350758 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1d1ce-c1ea-43cd-ac46-8d03b37ef571 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:44.351033 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1d1ce-c1ea-43cd-ac46-8d03b37ef571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:44.351299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1d1ce-c1ea-43cd-ac46-8d03b37ef571 admin admin] PATH_INFO: `/v3/domains` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:44.351935 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1d1ce-c1ea-43cd-ac46-8d03b37ef571 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81402) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 19:43:44.352230 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1d1ce-c1ea-43cd-ac46-8d03b37ef571 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:44.353826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1d1ce-c1ea-43cd-ac46-8d03b37ef571 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:44.365094 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1471/2903] 10.222.0.17 () {58 vars in 1153 bytes} [Fri Sep 30 19:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 19:43:44.369160 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca889dd3-1280-4533-8209-22ab273a70c4 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:44.378178 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca889dd3-1280-4533-8209-22ab273a70c4 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:44.382846 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa50075-529a-4884-b066-180ec00e044a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:44.383357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa50075-529a-4884-b066-180ec00e044a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:44.383639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa50075-529a-4884-b066-180ec00e044a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:44.383910 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa50075-529a-4884-b066-180ec00e044a admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:44.384560 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa50075-529a-4884-b066-180ec00e044a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:44.386454 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa50075-529a-4884-b066-180ec00e044a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:44.414559 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1433/2904] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:43:44 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:43:44.418813 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1d1ce-c1ea-43cd-ac46-8d03b37ef571 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:44.428595 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1d1ce-c1ea-43cd-ac46-8d03b37ef571 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:44.432050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:43:44.485999 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ede83-e738-49d9-b3ec-b52d7a5f2369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:44.486518 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ede83-e738-49d9-b3ec-b52d7a5f2369 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:44.486796 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ede83-e738-49d9-b3ec-b52d7a5f2369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:44.487091 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ede83-e738-49d9-b3ec-b52d7a5f2369 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:44.487719 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ede83-e738-49d9-b3ec-b52d7a5f2369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:44.489653 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ede83-e738-49d9-b3ec-b52d7a5f2369 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:44.509745 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1472/2905] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:43:44 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:43:44.514383 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa50075-529a-4884-b066-180ec00e044a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:44.524393 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa50075-529a-4884-b066-180ec00e044a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:44.529463 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83965f1-6d20-49db-bd32-260e1fe65992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:44.530052 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83965f1-6d20-49db-bd32-260e1fe65992 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:44.530329 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83965f1-6d20-49db-bd32-260e1fe65992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:44.530595 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83965f1-6d20-49db-bd32-260e1fe65992 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:44.531224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83965f1-6d20-49db-bd32-260e1fe65992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:44.532845 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83965f1-6d20-49db-bd32-260e1fe65992 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:44.543044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1434/2906] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:43:44.546777 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ede83-e738-49d9-b3ec-b52d7a5f2369 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:44.556323 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ede83-e738-49d9-b3ec-b52d7a5f2369 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:44.560826 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866caa77-89fe-467c-99d7-c51593198dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:44.561399 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866caa77-89fe-467c-99d7-c51593198dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:44.561673 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866caa77-89fe-467c-99d7-c51593198dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:44.561938 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866caa77-89fe-467c-99d7-c51593198dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:44.562566 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866caa77-89fe-467c-99d7-c51593198dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:44.564116 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866caa77-89fe-467c-99d7-c51593198dc6 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:44.574470 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1473/2907] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:43:44.578954 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83965f1-6d20-49db-bd32-260e1fe65992 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:44.588890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83965f1-6d20-49db-bd32-260e1fe65992 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:44.594001 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe097358-355c-42d8-8a13-5a68fba37537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:44.594636 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe097358-355c-42d8-8a13-5a68fba37537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:44.594915 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe097358-355c-42d8-8a13-5a68fba37537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:44.595184 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe097358-355c-42d8-8a13-5a68fba37537 admin admin] PATH_INFO: `/v3/projects/9b7a0b8124ba4b51bcb69cb5ff459819/users/ed107378a0a24db8962ac5da8c6dd961/roles/43452aaa2c9c4505b4bd3dd1b2412c65` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:44.610619 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe097358-355c-42d8-8a13-5a68fba37537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7a0b8124ba4b51bcb69cb5ff459819, user_id=ed107378a0a24db8962ac5da8c6dd961, role_id=43452aaa2c9c4505b4bd3dd1b2412c65)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:44.613281 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe097358-355c-42d8-8a13-5a68fba37537 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:44.625449 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1435/2908] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:43:44 2022] PUT /identity/v3/projects/9b7a0b8124ba4b51bcb69cb5ff459819/users/ed107378a0a24db8962ac5da8c6dd961/roles/43452aaa2c9c4505b4bd3dd1b2412c65 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:43:44.630355 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96d2ea-2661-4fde-9dc0-3b0a70fa569a None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:44.630636 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96d2ea-2661-4fde-9dc0-3b0a70fa569a None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:44.630904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96d2ea-2661-4fde-9dc0-3b0a70fa569a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:44.649710 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-ab96d2ea-2661-4fde-9dc0-3b0a70fa569a None None] MFA Rules not processed for user `02c8dc74a8e74fac884b7eaf15083780`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:43:44.650881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-ab96d2ea-2661-4fde-9dc0-3b0a70fa569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:44.707881 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1474/2909] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 19:43:44.717375 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe097358-355c-42d8-8a13-5a68fba37537 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:44.719027 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-fe097358-355c-42d8-8a13-5a68fba37537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:44.732781 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:43:44.777462 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe097358-355c-42d8-8a13-5a68fba37537 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:44.780895 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16230e-e94b-4010-bff4-85e4eeb1fc5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:44.781489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16230e-e94b-4010-bff4-85e4eeb1fc5e service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:44.781762 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16230e-e94b-4010-bff4-85e4eeb1fc5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:44.782029 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16230e-e94b-4010-bff4-85e4eeb1fc5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:44.790958 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16230e-e94b-4010-bff4-85e4eeb1fc5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:44.792377 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16230e-e94b-4010-bff4-85e4eeb1fc5e service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:44.801835 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1436/2910] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:43:48.930124 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6d533-2b6a-4775-ac9c-3013a86d1bdb None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:48.930696 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6d533-2b6a-4775-ac9c-3013a86d1bdb None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:48.930904 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6d533-2b6a-4775-ac9c-3013a86d1bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:48.962841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-bef6d533-2b6a-4775-ac9c-3013a86d1bdb None None] MFA Rules not processed for user `ed107378a0a24db8962ac5da8c6dd961`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:43:48.964067 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bef6d533-2b6a-4775-ac9c-3013a86d1bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:49.031020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1475/2911] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:43:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:43:49.035658 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16230e-e94b-4010-bff4-85e4eeb1fc5e service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:49.046637 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16230e-e94b-4010-bff4-85e4eeb1fc5e service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:49.051370 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf07abc-414c-4b52-9401-e2c366edd51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:49.051977 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf07abc-414c-4b52-9401-e2c366edd51a admin admin] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:49.052257 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf07abc-414c-4b52-9401-e2c366edd51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:49.052599 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf07abc-414c-4b52-9401-e2c366edd51a admin admin] PATH_INFO: `/v3/projects` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:49.053232 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf07abc-414c-4b52-9401-e2c366edd51a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:49.055186 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf07abc-414c-4b52-9401-e2c366edd51a admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:49.082865 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1437/2912] 10.222.0.17 () {60 vars in 1153 bytes} [Fri Sep 30 19:43:49 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 19:43:49.086934 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6d533-2b6a-4775-ac9c-3013a86d1bdb None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:49.097618 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6d533-2b6a-4775-ac9c-3013a86d1bdb None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:49.110377 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:43:49.157726 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee361deb-440a-4268-a995-b52d6458b566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:49.158339 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee361deb-440a-4268-a995-b52d6458b566 admin admin] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:49.158622 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee361deb-440a-4268-a995-b52d6458b566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:49.158902 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee361deb-440a-4268-a995-b52d6458b566 admin admin] PATH_INFO: `/v3/users` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:49.159534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee361deb-440a-4268-a995-b52d6458b566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:49.161762 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee361deb-440a-4268-a995-b52d6458b566 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:49.182960 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1476/2913] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Sep 30 19:43:49 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 19:43:49.187641 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf07abc-414c-4b52-9401-e2c366edd51a admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:49.198070 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf07abc-414c-4b52-9401-e2c366edd51a admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:49.202692 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec604d-6fa1-41b3-ae1a-2bf5caa53ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:49.203282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec604d-6fa1-41b3-ae1a-2bf5caa53ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:49.203565 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec604d-6fa1-41b3-ae1a-2bf5caa53ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:49.203836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec604d-6fa1-41b3-ae1a-2bf5caa53ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:49.204509 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec604d-6fa1-41b3-ae1a-2bf5caa53ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:49.206095 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec604d-6fa1-41b3-ae1a-2bf5caa53ab2 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:49.218799 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1438/2914] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:43:49.222851 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee361deb-440a-4268-a995-b52d6458b566 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:49.233255 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee361deb-440a-4268-a995-b52d6458b566 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:49.237751 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199beb8-9d90-4ba1-88cb-c3dad1fd712e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:49.238386 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199beb8-9d90-4ba1-88cb-c3dad1fd712e admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:49.238660 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199beb8-9d90-4ba1-88cb-c3dad1fd712e admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:49.238930 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199beb8-9d90-4ba1-88cb-c3dad1fd712e admin admin] PATH_INFO: `/v3/projects/deeb9be0cdd9466fbde029e213c04ba9/users/81307635c25e4cc89d31f533178c1db4/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:49.253832 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199beb8-9d90-4ba1-88cb-c3dad1fd712e admin admin] RBAC: Authorizing `identity:create_grant(project_id=deeb9be0cdd9466fbde029e213c04ba9, user_id=81307635c25e4cc89d31f533178c1db4, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:49.256628 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199beb8-9d90-4ba1-88cb-c3dad1fd712e admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:49.268780 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1477/2915] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:43:49 2022] PUT /identity/v3/projects/deeb9be0cdd9466fbde029e213c04ba9/users/81307635c25e4cc89d31f533178c1db4/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:43:49.272640 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec604d-6fa1-41b3-ae1a-2bf5caa53ab2 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:49.282961 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec604d-6fa1-41b3-ae1a-2bf5caa53ab2 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:49.337284 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd1621-8ade-4753-b248-d4346c90a521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:49.337975 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd1621-8ade-4753-b248-d4346c90a521 admin admin] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:49.338251 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd1621-8ade-4753-b248-d4346c90a521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:49.338518 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd1621-8ade-4753-b248-d4346c90a521 admin admin] PATH_INFO: `/v3/roles` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:49.339323 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd1621-8ade-4753-b248-d4346c90a521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:49.343524 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd1621-8ade-4753-b248-d4346c90a521 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:49.352691 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1439/2916] 10.222.0.17 () {58 vars in 1122 bytes} [Fri Sep 30 19:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 19:43:49.357656 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199beb8-9d90-4ba1-88cb-c3dad1fd712e admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:49.368806 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199beb8-9d90-4ba1-88cb-c3dad1fd712e admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:49.377169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb01ff-5234-4f46-a165-2b191a27a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:49.377849 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb01ff-5234-4f46-a165-2b191a27a769 admin admin] REQUEST_METHOD: `PUT` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:49.378127 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb01ff-5234-4f46-a165-2b191a27a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:49.378402 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb01ff-5234-4f46-a165-2b191a27a769 admin admin] PATH_INFO: `/v3/projects/deeb9be0cdd9466fbde029e213c04ba9/users/81307635c25e4cc89d31f533178c1db4/roles/6c2982cfde074f068473596a4f4e61c2` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:49.380344 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb01ff-5234-4f46-a165-2b191a27a769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deeb9be0cdd9466fbde029e213c04ba9, user_id=81307635c25e4cc89d31f533178c1db4, role_id=6c2982cfde074f068473596a4f4e61c2)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:49.383047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb01ff-5234-4f46-a165-2b191a27a769 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:49.401015 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1478/2917] 10.222.0.17 () {60 vars in 1483 bytes} [Fri Sep 30 19:43:49 2022] PUT /identity/v3/projects/deeb9be0cdd9466fbde029e213c04ba9/users/81307635c25e4cc89d31f533178c1db4/roles/6c2982cfde074f068473596a4f4e61c2 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:43:53.578256 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6c794-64f5-4b0c-8e9e-90ab376143ba None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:53.578778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6c794-64f5-4b0c-8e9e-90ab376143ba None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:53.578778 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6c794-64f5-4b0c-8e9e-90ab376143ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:53.615739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-11a6c794-64f5-4b0c-8e9e-90ab376143ba None None] MFA Rules not processed for user `81307635c25e4cc89d31f533178c1db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:43:53.617342 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-11a6c794-64f5-4b0c-8e9e-90ab376143ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:53.727242 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1440/2918] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:43:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:43:53.740208 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb01ff-5234-4f46-a165-2b191a27a769 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:53.742149 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-5ffb01ff-5234-4f46-a165-2b191a27a769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:53.809064 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb01ff-5234-4f46-a165-2b191a27a769 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:53.813952 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2683ac-9c18-43ec-9d40-35fdda042c13 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a9b371434243daa4176215daefb79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:53.814474 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2683ac-9c18-43ec-9d40-35fdda042c13 service glance] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:53.814747 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2683ac-9c18-43ec-9d40-35fdda042c13 service glance] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:53.815009 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2683ac-9c18-43ec-9d40-35fdda042c13 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:53.875741 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2683ac-9c18-43ec-9d40-35fdda042c13 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:53.877827 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2683ac-9c18-43ec-9d40-35fdda042c13 service glance] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:53.890957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1479/2919] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:43:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:43:54.047398 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7128ba7-db86-43b3-bb5b-5110a5d50813 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:54.047706 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7128ba7-db86-43b3-bb5b-5110a5d50813 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:54.047982 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7128ba7-db86-43b3-bb5b-5110a5d50813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:54.058852 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:43:54.073301 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-f7128ba7-db86-43b3-bb5b-5110a5d50813 None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:43:54.074592 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-f7128ba7-db86-43b3-bb5b-5110a5d50813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:54.141922 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1441/2920] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:43:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:43:54.197300 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2683ac-9c18-43ec-9d40-35fdda042c13 service glance] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:43:54.198836 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-9c2683ac-9c18-43ec-9d40-35fdda042c13 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:43:54.233132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:43:54.275922 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2683ac-9c18-43ec-9d40-35fdda042c13 service glance] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:43:54.281688 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed932b1-0230-42f3-b718-ecdba5bfa750 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:43:54.282290 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed932b1-0230-42f3-b718-ecdba5bfa750 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:43:54.282564 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed932b1-0230-42f3-b718-ecdba5bfa750 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:43:54.282841 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed932b1-0230-42f3-b718-ecdba5bfa750 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:43:54.291164 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed932b1-0230-42f3-b718-ecdba5bfa750 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:43:54.292919 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed932b1-0230-42f3-b718-ecdba5bfa750 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:43:54.301006 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1480/2921] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:43:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:44:02.961887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4157fd-31d7-4805-b9be-6b3b32968a6b None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:44:02.962380 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4157fd-31d7-4805-b9be-6b3b32968a6b None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:44:02.962486 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4157fd-31d7-4805-b9be-6b3b32968a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:44:02.981007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-4b4157fd-31d7-4805-b9be-6b3b32968a6b None None] MFA Rules not processed for user `ed107378a0a24db8962ac5da8c6dd961`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:44:02.982210 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-4b4157fd-31d7-4805-b9be-6b3b32968a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:44:02.988943 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1442/2922] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:44:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:44:03.028261 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed932b1-0230-42f3-b718-ecdba5bfa750 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:44:03.037697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed932b1-0230-42f3-b718-ecdba5bfa750 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:44:03.041306 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418fdaa-c227-42c6-b580-b56e5a25900c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a9b371434243daa4176215daefb79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:44:03.041918 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418fdaa-c227-42c6-b580-b56e5a25900c service glance] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:44:03.042193 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418fdaa-c227-42c6-b580-b56e5a25900c service glance] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:44:03.042459 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418fdaa-c227-42c6-b580-b56e5a25900c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:44:03.044405 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-a418fdaa-c227-42c6-b580-b56e5a25900c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:44:03.113778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418fdaa-c227-42c6-b580-b56e5a25900c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:44:03.115101 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418fdaa-c227-42c6-b580-b56e5a25900c service glance] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:44:03.126519 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1481/2923] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:44:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:44:03.658455 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4157fd-31d7-4805-b9be-6b3b32968a6b None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:44:03.715083 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4157fd-31d7-4805-b9be-6b3b32968a6b None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:44:03.719724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60e973-0485-4075-bd7b-b1252a3887d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:44:03.720511 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60e973-0485-4075-bd7b-b1252a3887d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:44:03.720932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60e973-0485-4075-bd7b-b1252a3887d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:44:03.724583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60e973-0485-4075-bd7b-b1252a3887d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:44:03.726418 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-da60e973-0485-4075-bd7b-b1252a3887d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:44:03.740497 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60e973-0485-4075-bd7b-b1252a3887d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:44:03.741864 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60e973-0485-4075-bd7b-b1252a3887d0 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:44:03.753873 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1443/2924] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:44:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:44:05.020420 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570262b-177f-4b1c-ab93-f78532dcd6b4 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:44:05.020910 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570262b-177f-4b1c-ab93-f78532dcd6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:44:05.021046 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570262b-177f-4b1c-ab93-f78532dcd6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:44:05.040982 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-f570262b-177f-4b1c-ab93-f78532dcd6b4 None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:44:05.042224 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-f570262b-177f-4b1c-ab93-f78532dcd6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:44:05.050602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1482/2925] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Sep 30 19:44:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:44:05.062369 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60e973-0485-4075-bd7b-b1252a3887d0 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:44:05.074045 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60e973-0485-4075-bd7b-b1252a3887d0 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:44:05.079032 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d61efa-9418-49dc-8f24-f40dca5b2968 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:44:05.079626 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d61efa-9418-49dc-8f24-f40dca5b2968 service swift] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:44:05.079902 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d61efa-9418-49dc-8f24-f40dca5b2968 service swift] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:44:05.080169 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d61efa-9418-49dc-8f24-f40dca5b2968 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:44:05.088303 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d61efa-9418-49dc-8f24-f40dca5b2968 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:44:05.089928 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d61efa-9418-49dc-8f24-f40dca5b2968 service swift] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:44:05.098568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1444/2926] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:44:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:44:15.034925 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=59 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:44:15.095175 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=58 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:44:36.320992 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570262b-177f-4b1c-ab93-f78532dcd6b4 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:44:36.330864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570262b-177f-4b1c-ab93-f78532dcd6b4 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:44:36.334376 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70851c22-dad7-4ca8-88c7-6e78c0a7c399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:44:36.334964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851c22-dad7-4ca8-88c7-6e78c0a7c399 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:44:36.335235 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851c22-dad7-4ca8-88c7-6e78c0a7c399 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:44:36.335501 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851c22-dad7-4ca8-88c7-6e78c0a7c399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:44:36.344717 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70851c22-dad7-4ca8-88c7-6e78c0a7c399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:44:36.346158 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70851c22-dad7-4ca8-88c7-6e78c0a7c399 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:44:36.355326 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1483/2927] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:44:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:44:43.262215 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d61efa-9418-49dc-8f24-f40dca5b2968 service swift] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:44:43.263588 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-81d61efa-9418-49dc-8f24-f40dca5b2968 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:44:43.319436 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d61efa-9418-49dc-8f24-f40dca5b2968 service swift] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:44:43.322944 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee5d12-8cc3-4566-a3ec-06094432fb11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d3020e1f6c414caabbaa185fb7ed48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:44:43.323530 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee5d12-8cc3-4566-a3ec-06094432fb11 service cinder] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:44:43.323801 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee5d12-8cc3-4566-a3ec-06094432fb11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:44:43.324061 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee5d12-8cc3-4566-a3ec-06094432fb11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:44:43.325823 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-c6ee5d12-8cc3-4566-a3ec-06094432fb11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:44:43.384594 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee5d12-8cc3-4566-a3ec-06094432fb11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:44:43.386049 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee5d12-8cc3-4566-a3ec-06094432fb11 service cinder] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:44:43.395892 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1445/2928] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:44:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:44:46.351297 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=6 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:44:53.392070 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=48 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:44:54.403136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e6fdd-787a-411f-acb1-df9cf2b94195 None None] REQUEST_METHOD: `POST` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:44:54.403136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e6fdd-787a-411f-acb1-df9cf2b94195 None None] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:44:54.403136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e6fdd-787a-411f-acb1-df9cf2b94195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:44:54.421669 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.auth.core [None req-0c3e6fdd-787a-411f-acb1-df9cf2b94195 None None] MFA Rules not processed for user `eccbf92faf2b4bad8746a2a48f5a6226`. Rule list: `[]` (Enabled: `True`). {{(pid=81402) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:44:54.422838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0c3e6fdd-787a-411f-acb1-df9cf2b94195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:44:54.429527 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1484/2929] 10.222.0.17 () {58 vars in 958 bytes} [Fri Sep 30 19:44:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:45:04.416466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=3 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:45:13.014639 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee5d12-8cc3-4566-a3ec-06094432fb11 service cinder] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:45:13.023811 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee5d12-8cc3-4566-a3ec-06094432fb11 service cinder] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:45:13.027439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e938f50-667f-4d9e-9e49-85bcb5807736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:45:13.028213 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e938f50-667f-4d9e-9e49-85bcb5807736 service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:45:13.028535 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e938f50-667f-4d9e-9e49-85bcb5807736 service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:45:13.028813 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e938f50-667f-4d9e-9e49-85bcb5807736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:45:13.030583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7e938f50-667f-4d9e-9e49-85bcb5807736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:45:13.092489 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e938f50-667f-4d9e-9e49-85bcb5807736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:45:13.093921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e938f50-667f-4d9e-9e49-85bcb5807736 service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:45:13.104025 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1446/2930] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:45:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:45:23.099546 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=30 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:45:24.258964 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e6fdd-787a-411f-acb1-df9cf2b94195 None None] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:45:24.260741 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-0c3e6fdd-787a-411f-acb1-df9cf2b94195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:45:24.272701 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e6fdd-787a-411f-acb1-df9cf2b94195 None None] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:45:24.276161 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbedf0-4de6-48cf-9c01-179f463c5218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:45:24.276778 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbedf0-4de6-48cf-9c01-179f463c5218 service neutron] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:45:24.277264 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbedf0-4de6-48cf-9c01-179f463c5218 service neutron] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:45:24.277538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbedf0-4de6-48cf-9c01-179f463c5218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:45:24.279169 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-75fbedf0-4de6-48cf-9c01-179f463c5218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:45:24.288968 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fbedf0-4de6-48cf-9c01-179f463c5218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:45:24.290374 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fbedf0-4de6-48cf-9c01-179f463c5218 service neutron] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:45:24.298210 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1485/2931] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:45:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:45:34.294665 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=6 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:47:06.222519 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e938f50-667f-4d9e-9e49-85bcb5807736 service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:06.224934 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-7e938f50-667f-4d9e-9e49-85bcb5807736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:47:06.281389 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e938f50-667f-4d9e-9e49-85bcb5807736 service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:06.287932 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1d526-92f9-4cbc-bd42-2a8e19e35b81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:06.288576 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1d526-92f9-4cbc-bd42-2a8e19e35b81 service nova] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:06.288858 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1d526-92f9-4cbc-bd42-2a8e19e35b81 service nova] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:06.289140 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1d526-92f9-4cbc-bd42-2a8e19e35b81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:06.290903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-16d1d526-92f9-4cbc-bd42-2a8e19e35b81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:47:06.304264 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1d526-92f9-4cbc-bd42-2a8e19e35b81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:06.306064 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1d526-92f9-4cbc-bd42-2a8e19e35b81 service nova] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:06.318727 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1447/2932] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:47:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:47:16.313836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=21 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:47:18.448705 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbedf0-4de6-48cf-9c01-179f463c5218 service neutron] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:18.475266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbedf0-4de6-48cf-9c01-179f463c5218 service neutron] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:18.480712 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f90549-7a34-47ac-b6af-96dbb28f9417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5952db29e6fc4ddbb9aeb0522497312c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:18.481307 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f90549-7a34-47ac-b6af-96dbb28f9417 service nova] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:18.481602 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f90549-7a34-47ac-b6af-96dbb28f9417 service nova] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:18.481871 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f90549-7a34-47ac-b6af-96dbb28f9417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:18.483572 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-40f90549-7a34-47ac-b6af-96dbb28f9417 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:47:18.506847 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f90549-7a34-47ac-b6af-96dbb28f9417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:18.508461 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f90549-7a34-47ac-b6af-96dbb28f9417 service nova] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:18.519284 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1486/2933] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:47:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:47:19.295568 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c7505-eeec-4710-9e23-e87841dddb30 None None] REQUEST_METHOD: `POST` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:19.296056 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c7505-eeec-4710-9e23-e87841dddb30 None None] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:19.296185 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c7505-eeec-4710-9e23-e87841dddb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:19.315475 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.auth.core [None req-e29c7505-eeec-4710-9e23-e87841dddb30 None None] MFA Rules not processed for user `b638b8149ae149e3aad92ed51be92588`. Rule list: `[]` (Enabled: `True`). {{(pid=81403) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 19:47:19.317051 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e29c7505-eeec-4710-9e23-e87841dddb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:47:19.324253 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1448/2934] 10.222.0.17 () {58 vars in 967 bytes} [Fri Sep 30 19:47:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 19:47:19.372266 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f90549-7a34-47ac-b6af-96dbb28f9417 service nova] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:19.381497 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f90549-7a34-47ac-b6af-96dbb28f9417 service nova] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:19.391055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae10c62-194c-4113-a6ca-7051d1b750c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8785c8c5ac84393afcaf88e891f3b7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:19.391792 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae10c62-194c-4113-a6ca-7051d1b750c1 service swift] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:19.392150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae10c62-194c-4113-a6ca-7051d1b750c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:19.392525 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae10c62-194c-4113-a6ca-7051d1b750c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:19.404394 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae10c62-194c-4113-a6ca-7051d1b750c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:19.406131 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae10c62-194c-4113-a6ca-7051d1b750c1 service swift] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:19.419188 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1487/2935] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Sep 30 19:47:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 19:47:20.828291 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c7505-eeec-4710-9e23-e87841dddb30 None None] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:20.838611 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c7505-eeec-4710-9e23-e87841dddb30 None None] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:20.846060 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36c1f4-6923-4137-a7ce-654009d4a7fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ea49d1f6c040f18511535b7e51a53b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:20.846669 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36c1f4-6923-4137-a7ce-654009d4a7fa service neutron] REQUEST_METHOD: `GET` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:20.846948 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36c1f4-6923-4137-a7ce-654009d4a7fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:20.847224 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36c1f4-6923-4137-a7ce-654009d4a7fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:20.849911 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-6b36c1f4-6923-4137-a7ce-654009d4a7fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:47:20.939260 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36c1f4-6923-4137-a7ce-654009d4a7fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:20.940739 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36c1f4-6923-4137-a7ce-654009d4a7fa service neutron] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:20.959903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1449/2936] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:47:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:47:27.638220 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae10c62-194c-4113-a6ca-7051d1b750c1 service swift] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:27.653736 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae10c62-194c-4113-a6ca-7051d1b750c1 service swift] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:27.669538 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:27.670503 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:27.671111 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:27.672128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] PATH_INFO: `/v3/users/ed107378a0a24db8962ac5da8c6dd961` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:27.674260 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed107378a0a24db8962ac5da8c6dd961)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:27.676783 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:27.734117 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed107378a0a24db8962ac5da8c6dd961'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:27.743074 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed107378a0a24db8962ac5da8c6dd961'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:27.743853 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed107378a0a24db8962ac5da8c6dd961'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:27.753332 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed107378a0a24db8962ac5da8c6dd961'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:27.769671 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed107378a0a24db8962ac5da8c6dd961'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:27.776697 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1488/2937] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:47:27 2022] DELETE /identity/v3/users/ed107378a0a24db8962ac5da8c6dd961 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:47:28.061041 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36c1f4-6923-4137-a7ce-654009d4a7fa service neutron] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:28.081214 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36c1f4-6923-4137-a7ce-654009d4a7fa service neutron] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:28.148719 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:28.150118 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:28.150583 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:28.151013 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] PATH_INFO: `/v3/projects/9b7a0b8124ba4b51bcb69cb5ff459819` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:28.152315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b7a0b8124ba4b51bcb69cb5ff459819)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:28.154510 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:28.214439 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b7a0b8124ba4b51bcb69cb5ff459819', 'request_id': 'req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.221709 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b7a0b8124ba4b51bcb69cb5ff459819', 'request_id': 'req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.224921 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:47:28.231387 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b7a0b8124ba4b51bcb69cb5ff459819', 'request_id': 'req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.246271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] The token cache is being invalidate because project 9b7a0b8124ba4b51bcb69cb5ff459819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:47:28.246564 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.247196 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.248717 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1450/2938] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:47:28 2022] DELETE /identity/v3/projects/9b7a0b8124ba4b51bcb69cb5ff459819 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:47:28.252709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:28.254299 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:47:28.268050 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c633be3-8b7e-40ac-b59d-85ac1ba9e0e5 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:28.335534 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:28.336095 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:28.336393 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:28.336667 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] PATH_INFO: `/v3/users/81307635c25e4cc89d31f533178c1db4` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:28.338760 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81307635c25e4cc89d31f533178c1db4)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:28.341947 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:28.378115 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:47:28.403803 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81307635c25e4cc89d31f533178c1db4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.414257 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81307635c25e4cc89d31f533178c1db4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.415030 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81307635c25e4cc89d31f533178c1db4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.425020 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81307635c25e4cc89d31f533178c1db4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.437174 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81307635c25e4cc89d31f533178c1db4'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.442973 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1489/2939] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:47:28 2022] DELETE /identity/v3/users/81307635c25e4cc89d31f533178c1db4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:47:28.654063 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:28.655538 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:47:28.679804 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d05d8-e84a-4c5e-9b49-4a0dfcb5e7f3 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:28.755720 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:28.756724 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:28.757271 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:28.757836 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] PATH_INFO: `/v3/projects/deeb9be0cdd9466fbde029e213c04ba9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:28.759277 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deeb9be0cdd9466fbde029e213c04ba9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:28.761374 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:28.820841 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deeb9be0cdd9466fbde029e213c04ba9', 'request_id': 'req-e2af6c00-c259-4a66-990f-e7bf63154dc0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.827007 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deeb9be0cdd9466fbde029e213c04ba9', 'request_id': 'req-e2af6c00-c259-4a66-990f-e7bf63154dc0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.834229 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deeb9be0cdd9466fbde029e213c04ba9', 'request_id': 'req-e2af6c00-c259-4a66-990f-e7bf63154dc0'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.850888 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] The token cache is being invalidate because project deeb9be0cdd9466fbde029e213c04ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:47:28.851120 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.851805 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:28.856505 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1451/2940] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:47:28 2022] DELETE /identity/v3/projects/deeb9be0cdd9466fbde029e213c04ba9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:47:28.892132 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:28.895924 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:47:28.919004 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b071287-9924-415d-9f5a-f7331370fc99 admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:28.997709 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:28.998360 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:28.998640 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:28.998915 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] PATH_INFO: `/v3/users/eccbf92faf2b4bad8746a2a48f5a6226` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:29.000136 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eccbf92faf2b4bad8746a2a48f5a6226)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:29.002147 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:29.057145 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eccbf92faf2b4bad8746a2a48f5a6226'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.065302 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eccbf92faf2b4bad8746a2a48f5a6226'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.065962 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eccbf92faf2b4bad8746a2a48f5a6226'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.074373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eccbf92faf2b4bad8746a2a48f5a6226'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.087916 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eccbf92faf2b4bad8746a2a48f5a6226'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.095143 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1490/2941] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:47:28 2022] DELETE /identity/v3/users/eccbf92faf2b4bad8746a2a48f5a6226 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:47:29.267418 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:29.268890 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:47:29.281974 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af6c00-c259-4a66-990f-e7bf63154dc0 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:29.338058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:29.338686 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:29.338967 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:29.339243 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] PATH_INFO: `/v3/projects/267f4c2e27aa4f9db62c3877d948f3d9` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:29.340357 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=267f4c2e27aa4f9db62c3877d948f3d9)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:29.342282 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:29.364756 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:47:29.393220 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267f4c2e27aa4f9db62c3877d948f3d9', 'request_id': 'req-e4128209-f33c-4665-9725-7d776f3e60f5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.399887 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267f4c2e27aa4f9db62c3877d948f3d9', 'request_id': 'req-e4128209-f33c-4665-9725-7d776f3e60f5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.406157 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267f4c2e27aa4f9db62c3877d948f3d9', 'request_id': 'req-e4128209-f33c-4665-9725-7d776f3e60f5'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.418476 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] The token cache is being invalidate because project 267f4c2e27aa4f9db62c3877d948f3d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:47:29.418755 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.419409 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.420903 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1452/2942] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:47:29 2022] DELETE /identity/v3/projects/267f4c2e27aa4f9db62c3877d948f3d9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:47:29.424894 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:29.426415 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:47:29.439212 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c2a55-0476-4b4c-a3aa-cb7823d5172d admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:29.459416 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:47:29.498830 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:29.499466 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:29.499741 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:29.500007 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] PATH_INFO: `/v3/users/f70d52c9129c45e0b6b6e4a88c3d793c` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:29.501150 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f70d52c9129c45e0b6b6e4a88c3d793c)` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:29.503047 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:29.555411 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f70d52c9129c45e0b6b6e4a88c3d793c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.562268 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f70d52c9129c45e0b6b6e4a88c3d793c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.562929 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f70d52c9129c45e0b6b6e4a88c3d793c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.571373 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f70d52c9129c45e0b6b6e4a88c3d793c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.583680 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.notifications [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f70d52c9129c45e0b6b6e4a88c3d793c'} {{(pid=81402) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.589473 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1491/2943] 10.222.0.17 () {58 vars in 1224 bytes} [Fri Sep 30 19:47:29 2022] DELETE /identity/v3/users/f70d52c9129c45e0b6b6e4a88c3d793c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:47:29.760044 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] Authenticating user token {{(pid=81403) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:47:29.761596 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.fernet_utils [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81403) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:47:29.780332 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4128209-f33c-4665-9725-7d776f3e60f5 admin admin] Validating token access rules against request {{(pid=81403) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:47:29.838554 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02c8dc74a8e74fac884b7eaf15083780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e574e273dabe4a3eab67dc95d42fcb6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81403) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:47:29.839174 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:47:29.839447 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:47:29.839711 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] PATH_INFO: `/v3/projects/1a1b2e39947444958f8dd829153d3a21` {{(pid=81403) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:47:29.840827 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a1b2e39947444958f8dd829153d3a21)` {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:47:29.842705 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] RBAC: Authorization granted {{(pid=81403) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:47:29.887335 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a1b2e39947444958f8dd829153d3a21', 'request_id': 'req-3dd7a330-3175-4a73-860a-f51bad841237'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.892810 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a1b2e39947444958f8dd829153d3a21', 'request_id': 'req-3dd7a330-3175-4a73-860a-f51bad841237'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.899315 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a1b2e39947444958f8dd829153d3a21', 'request_id': 'req-3dd7a330-3175-4a73-860a-f51bad841237'} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.912058 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] The token cache is being invalidate because project 1a1b2e39947444958f8dd829153d3a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81403) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 19:47:29.912574 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.913211 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG keystone.notifications [None req-3dd7a330-3175-4a73-860a-f51bad841237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81403) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 19:47:29.914657 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: [pid: 81403|app: 0|req: 1453/2944] 10.222.0.17 () {58 vars in 1233 bytes} [Fri Sep 30 19:47:29 2022] DELETE /identity/v3/projects/1a1b2e39947444958f8dd829153d3a21 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 19:47:39.586055 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:47:39.908445 dv-cinder-860015-1-iscsi devstack@keystone.service[81403]: DEBUG dbcounter [-] [81403] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=81403) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 19:48:24.558105 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] Authenticating user token {{(pid=81402) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 19:48:24.559475 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:48:24.646838 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b57b1-9e3c-4a6b-bf2e-48655cff945f admin admin] Validating token access rules against request {{(pid=81402) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 19:48:24.703957 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2c52c-aa0e-47c4-af97-82e1d0f91717 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a50fd417ae94859bb713b8b2066a9f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e761ae9e97264607ad7eab7dc2e64b00', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81402) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 19:48:24.704586 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2c52c-aa0e-47c4-af97-82e1d0f91717 service placement] REQUEST_METHOD: `GET` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 19:48:24.704862 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2c52c-aa0e-47c4-af97-82e1d0f91717 service placement] SCRIPT_NAME: `/identity` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 19:48:24.705128 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2c52c-aa0e-47c4-af97-82e1d0f91717 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81402) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 19:48:24.706864 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.fernet_utils [None req-bbf2c52c-aa0e-47c4-af97-82e1d0f91717 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81402) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 19:48:24.717782 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2c52c-aa0e-47c4-af97-82e1d0f91717 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 19:48:24.719351 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2c52c-aa0e-47c4-af97-82e1d0f91717 service placement] RBAC: Authorization granted {{(pid=81402) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 19:48:24.729625 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: [pid: 81402|app: 0|req: 1492/2945] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Sep 30 19:48:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 19:48:34.725331 dv-cinder-860015-1-iscsi devstack@keystone.service[81402]: DEBUG dbcounter [-] [81402] Writing DB stats keystone:SELECT=67 {{(pid=81402) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}